2024/11/23-11:10:22.429149 139957290698304 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00: fork to child time = 0.007281 sec 2024/11/23-11:10:22.429750 139957290698304 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00 2024/11/23-11:10:22.436673 139957290698304 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00 2024/11/23-11:10:22.437555 139957290698304 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-11:10:22.437782 139957290698304 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-11:10:22.492767 139957290698304 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-24943/att-00/MANIFEST-206347 2024/11/23-11:10:22.525848 139957290698304 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-24943/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 257246, last_sequence is 3945495286, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 257034 2024/11/23-11:10:22.525854 139957290698304 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-11:10:22.525954 139957290698304 [version_set.cc:5721] Creating manifest 257246 2024/11/23-11:10:22.561118 139957290698304 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-11:10:22.582703 139957290698304 [compaction_job.cc:2437] [default] [JOB 24943] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-11:10:22.582717 139957290698304 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 18702 Base level 2, inputs: [257175(27MB)], [257233(8142KB) 257116(34MB) 257136(36MB) 257137(35MB) 257138(33MB) 257139(33MB) 257140(35MB) 257141(36MB) 257142(33MB) 257143(36MB) 257144(35MB) 257145(33MB) 257146(36MB) 257147(31MB) 257234(33MB)] 2024/11/23-11:10:22.589350 139957290698304 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-11:10:22.589377 139957290698304 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-11:10:22.589395 139957290698304 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-11:10:22.589400 139957290698304 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-11:10:22.589411 139957290698304 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-11:10:22.990314 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257247.sst) /dev/shm/ToplingTemp/Topling-33512-1dGxdf 2024/11/23-11:10:24.010162 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257247.sst): first pass time = 1.02's, 131.611'MB/sec 2024/11/23-11:10:25.331695 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257247.sst): old prealloc_size = 147639500, real_size = 32808368 2024/11/23-11:10:25.332265 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257247.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930953 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.17 avg-zval = 33.08 2024/11/23-11:10:25.333129 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257247.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:10:26.191618 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257247.sst): second pass time = 1.32's, 81.891'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.068'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3248.225'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 291.640'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33139.706'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.891'MB/sec zip pipeline throughput = 81.891'MB/sec entries = 930953 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.17 avg-zval = 33.08 usrkeys = 930953 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.17 avg-zval = 33.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930953 bytes = 3374720 } seq expand size = 6516671 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4442 value = 3.5122 dict = 1.00 all = 3.6241 } Zip/UnZip{ index = 0.0957 value = 0.2847 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 8.963288 GB avg = 0.168 KB (by entry num) total key len = 1.466888 GB avg = 0.028 KB total ukey len = 1.016497 GB avg = 0.015 KB total ukey num = 0.066481388 Billion total entry num = 0.053289117 Billion write speed all = 1.285457841 MB/sec (with seq num) write speed all = 1.232917276 MB/sec (without seq num) 2024/11/23-11:10:26.236862 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257247: 930953 keys, 37034838 bytes, temperature: kWarm 2024/11/23-11:10:26.236959 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331426236898, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257247, "file_size": 37034838, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3945177978, "table_properties": {"data_size": 30792736, "index_size": 1782720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26066684, "raw_average_key_size": 28, "raw_value_size": 108151082, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930953, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331422, "oldest_key_time": 0, "file_creation_time": 1732331422, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257247, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:10:26.272574 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257248.sst) /dev/shm/ToplingTemp/Topling-33512-nqrXKd 2024/11/23-11:10:27.314842 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257248.sst): first pass time = 1.04's, 128.779'MB/sec 2024/11/23-11:10:28.629858 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257248.sst): old prealloc_size = 147639500, real_size = 32514192 2024/11/23-11:10:28.630451 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257248.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 938670 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1079 GB zip-val = 0.0305 GB avg-val = 114.99 avg-zval = 32.47 2024/11/23-11:10:28.631281 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257248.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:10:29.480440 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257248.sst): second pass time = 1.31's, 82.129'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.361'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3135.963'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 292.523'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.076'MB, throughput = 34662.373'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.129'MB/sec zip pipeline throughput = 82.129'MB/sec entries = 938670 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.99 avg-zval = 32.47 usrkeys = 938670 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.99 avg-zval = 32.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 938670 bytes = 3520032 } seq expand size = 6570690 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4445 value = 3.5410 dict = 1.00 all = 3.6391 } Zip/UnZip{ index = 0.0957 value = 0.2824 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 9.077793 GB avg = 0.167 KB (by entry num) total key len = 1.493171 GB avg = 0.028 KB total ukey len = 1.035270 GB avg = 0.015 KB total ukey num = 0.067420058 Billion total entry num = 0.054227787 Billion write speed all = 1.302281337 MB/sec (with seq num) write speed all = 1.248836949 MB/sec (without seq num) 2024/11/23-11:10:29.527327 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257248: 938670 keys, 36881792 bytes, temperature: kWarm 2024/11/23-11:10:29.527393 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331429527358, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257248, "file_size": 36881792, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3945177724, "table_properties": {"data_size": 30481920, "index_size": 1797440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26282760, "raw_average_key_size": 28, "raw_value_size": 107934988, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 938670, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331426, "oldest_key_time": 0, "file_creation_time": 1732331426, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257248, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:10:29.609142 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257249.sst) /dev/shm/ToplingTemp/Topling-33512-ciSNGe 2024/11/23-11:10:30.503447 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257249.sst): first pass time = 0.89's, 150.088'MB/sec 2024/11/23-11:10:31.799087 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257249.sst): old prealloc_size = 147639500, real_size = 30777440 2024/11/23-11:10:31.799624 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257249.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930255 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0288 GB avg-val = 116.28 avg-zval = 30.92 2024/11/23-11:10:31.800439 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257249.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:10:32.620514 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257249.sst): second pass time = 1.29's, 83.542'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.612'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3459.554'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.335'MB/sec write SST data time = 0.13's, 278.155'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32856.864'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.542'MB/sec zip pipeline throughput = 83.542'MB/sec entries = 930255 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.28 avg-zval = 30.92 usrkeys = 930255 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.28 avg-zval = 30.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930255 bytes = 3372192 } seq expand size = 6511785 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4442 value = 3.7607 dict = 1.00 all = 3.8346 } Zip/UnZip{ index = 0.0957 value = 0.2659 dict = 1.00 all = 0.2608 } ---------------------------- total value len = 9.192476 GB avg = 0.167 KB (by entry num) total key len = 1.519218 GB avg = 0.028 KB total ukey len = 1.053876 GB avg = 0.015 KB total ukey num = 0.068350313 Billion total entry num = 0.055158042 Billion write speed all = 1.319108122 MB/sec (with seq num) write speed all = 1.264767939 MB/sec (without seq num) 2024/11/23-11:10:32.663034 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257249: 930255 keys, 35001554 bytes, temperature: kWarm 2024/11/23-11:10:32.663101 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331432663066, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257249, "file_size": 35001554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 3945176523, "table_properties": {"data_size": 28763344, "index_size": 1781376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26047140, "raw_average_key_size": 28, "raw_value_size": 108170650, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930255, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331429, "oldest_key_time": 0, "file_creation_time": 1732331429, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257249, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:10:32.701859 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257250.sst) /dev/shm/ToplingTemp/Topling-33512-TPVtDd 2024/11/23-11:10:33.712320 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257250.sst): first pass time = 1.01's, 132.833'MB/sec 2024/11/23-11:10:34.990379 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257250.sst): old prealloc_size = 147639500, real_size = 30870704 2024/11/23-11:10:34.990903 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257250.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925935 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 116.95 avg-zval = 31.17 2024/11/23-11:10:34.991714 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257250.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:10:35.808556 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257250.sst): second pass time = 1.28's, 84.780'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.736'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3533.519'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 279.715'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36972.905'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.780'MB/sec zip pipeline throughput = 84.780'MB/sec entries = 925935 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.95 avg-zval = 31.17 usrkeys = 925935 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.95 avg-zval = 31.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925935 bytes = 3240784 } seq expand size = 6481545 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4441 value = 3.7515 dict = 1.00 all = 3.8387 } Zip/UnZip{ index = 0.0957 value = 0.2666 dict = 1.00 all = 0.2605 } ---------------------------- total value len = 9.307249 GB avg = 0.166 KB (by entry num) total key len = 1.545144 GB avg = 0.028 KB total ukey len = 1.072394 GB avg = 0.015 KB total ukey num = 0.069276248 Billion total entry num = 0.056083977 Billion write speed all = 1.335910293 MB/sec (with seq num) write speed all = 1.280679589 MB/sec (without seq num) 2024/11/23-11:10:35.852187 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257250: 925935 keys, 34964702 bytes, temperature: kWarm 2024/11/23-11:10:35.852259 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331435852220, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257250, "file_size": 34964702, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3731531312, "largest_seqno": 3945176015, "table_properties": {"data_size": 28865888, "index_size": 1773120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25926180, "raw_average_key_size": 28, "raw_value_size": 108291670, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925935, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331432, "oldest_key_time": 0, "file_creation_time": 1732331432, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257250, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:10:35.893404 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257251.sst) /dev/shm/ToplingTemp/Topling-33512-6YPxSb 2024/11/23-11:10:36.889319 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257251.sst): first pass time = 1.00's, 134.774'MB/sec 2024/11/23-11:10:38.204711 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257251.sst): old prealloc_size = 147639500, real_size = 32712944 2024/11/23-11:10:38.205254 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257251.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924313 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.21 avg-zval = 33.24 2024/11/23-11:10:38.206060 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257251.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:10:39.037395 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257251.sst): second pass time = 1.31's, 82.414'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.201'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3377.162'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.12's, 298.505'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31046.099'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.414'MB/sec zip pipeline throughput = 82.414'MB/sec entries = 924313 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.21 avg-zval = 33.24 usrkeys = 924313 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.21 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924313 bytes = 3235104 } seq expand size = 6470191 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5143 value = 3.5262 dict = 1.00 all = 3.6468 } Zip/UnZip{ index = 0.0951 value = 0.2836 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 9.422056 GB avg = 0.165 KB (by entry num) total key len = 1.571025 GB avg = 0.028 KB total ukey len = 1.090881 GB avg = 0.016 KB total ukey num = 0.070200561 Billion total entry num = 0.057008290 Billion write speed all = 1.352691097 MB/sec (with seq num) write speed all = 1.296572447 MB/sec (without seq num) 2024/11/23-11:10:39.084272 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257251: 924313 keys, 36804416 bytes, temperature: kWarm 2024/11/23-11:10:39.084355 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331439084319, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257251, "file_size": 36804416, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 3945178727, "table_properties": {"data_size": 30723488, "index_size": 1758208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25880764, "raw_average_key_size": 28, "raw_value_size": 108337090, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924313, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331435, "oldest_key_time": 0, "file_creation_time": 1732331435, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257251, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:10:39.120158 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257252.sst) /dev/shm/ToplingTemp/Topling-33512-WCBgZf 2024/11/23-11:10:40.121910 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257252.sst): first pass time = 1.00's, 133.988'MB/sec 2024/11/23-11:10:41.484741 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257252.sst): old prealloc_size = 147639500, real_size = 34621472 2024/11/23-11:10:41.485281 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257252.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934622 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0326 GB avg-val = 115.61 avg-zval = 34.88 2024/11/23-11:10:41.485848 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257252.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:10:42.285866 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257252.sst): second pass time = 1.36's, 79.373'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 47.052'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3381.021'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.346'MB/sec write SST data time = 0.12's, 302.266'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32487.490'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.373'MB/sec zip pipeline throughput = 79.373'MB/sec entries = 934622 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.61 avg-zval = 34.88 usrkeys = 934622 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.04 avg-zval = 34.88 TagRS{ kinds = RS_Key0_TagN bytes = 146120 } TagArray{ size = 591957 bytes = 2071872 } seq expand size = 4143699 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4449 value = 3.3146 dict = 1.00 all = 3.5608 } Zip/UnZip{ index = 0.0957 value = 0.3017 dict = 1.00 all = 0.2808 } ---------------------------- total value len = 9.534248 GB avg = 0.165 KB (by entry num) total key len = 1.597194 GB avg = 0.028 KB total ukey len = 1.109573 GB avg = 0.016 KB total ukey num = 0.071135183 Billion total entry num = 0.057942912 Billion write speed all = 1.369169088 MB/sec (with seq num) write speed all = 1.312153192 MB/sec (without seq num) 2024/11/23-11:10:42.332944 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257252: 934622 keys, 37693308 bytes, temperature: kWarm 2024/11/23-11:10:42.333025 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331442332989, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257252, "file_size": 37693308, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3945178113, "table_properties": {"data_size": 32598032, "index_size": 1789632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26169416, "raw_average_key_size": 28, "raw_value_size": 108048316, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934622, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331439, "oldest_key_time": 0, "file_creation_time": 1732331439, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257252, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:10:42.372343 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257253.sst) /dev/shm/ToplingTemp/Topling-33512-BwUwBf 2024/11/23-11:10:43.429984 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257253.sst): first pass time = 1.06's, 126.907'MB/sec 2024/11/23-11:10:44.808501 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257253.sst): old prealloc_size = 147639500, real_size = 35259840 2024/11/23-11:10:44.809188 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257253.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081299 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0329 GB avg-val = 96.13 avg-zval = 30.44 2024/11/23-11:10:44.809367 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257253.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:10:45.487502 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257253.sst): second pass time = 1.38's, 75.446'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.68's, 53.468'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3118.273'MB/sec (index lex order gen) rebuild zvType time = 0.54's, 0.498'MB/sec write SST data time = 0.14's, 266.959'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.032'MB, throughput = 34003.428'MB/sec zip my value time = 1.38's, unzip length = 0.104'GB zip my value throughput = 75.446'MB/sec zip pipeline throughput = 75.446'MB/sec entries = 1081299 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.13 avg-zval = 30.44 usrkeys = 1081299 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.36 avg-zval = 30.44 TagRS{ kinds = RS_Key0_TagN bytes = 169040 } TagArray{ size = 35561 bytes = 102256 } seq expand size = 248927 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0329 GB dict = 1.03 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4479 value = 3.1574 dict = 1.00 all = 3.6971 } Zip/UnZip{ index = 0.0957 value = 0.3167 dict = 1.00 all = 0.2705 } ---------------------------- total value len = 9.638439 GB avg = 0.163 KB (by entry num) total key len = 1.627470 GB avg = 0.028 KB total ukey len = 1.131199 GB avg = 0.016 KB total ukey num = 0.072216482 Billion total entry num = 0.059024211 Billion write speed all = 1.385163039 MB/sec (with seq num) write speed all = 1.327106006 MB/sec (without seq num) 2024/11/23-11:10:45.532730 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257253: 1081299 keys, 36303577 bytes, temperature: kWarm 2024/11/23-11:10:45.532801 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331445532765, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257253, "file_size": 36303577, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3280379164, "table_properties": {"data_size": 32919472, "index_size": 2069888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30276372, "raw_average_key_size": 28, "raw_value_size": 103941447, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081299, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331442, "oldest_key_time": 0, "file_creation_time": 1732331442, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257253, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:10:45.568272 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257254.sst) /dev/shm/ToplingTemp/Topling-33512-cCLsSb 2024/11/23-11:10:46.502252 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257254.sst): first pass time = 0.93's, 143.712'MB/sec 2024/11/23-11:10:47.796943 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257254.sst): old prealloc_size = 147639500, real_size = 31486768 2024/11/23-11:10:47.797504 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257254.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.09 2024/11/23-11:10:47.798340 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257254.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:10:48.610089 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257254.sst): second pass time = 1.29's, 83.839'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 44.020'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.207'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.12's, 286.558'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 37031.058'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.839'MB/sec zip pipeline throughput = 83.839'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 32.09 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3447392 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0358 GB } UnZip/Zip{ index = 10.4428 value = 3.6777 dict = 1.00 all = 3.7498 } Zip/UnZip{ index = 0.0958 value = 0.2719 dict = 1.00 all = 0.2667 } ---------------------------- total value len = 9.753351 GB avg = 0.163 KB (by entry num) total key len = 1.653211 GB avg = 0.028 KB total ukey len = 1.149585 GB avg = 0.016 KB total ukey num = 0.073135782 Billion total entry num = 0.059943511 Billion write speed all = 1.401918322 MB/sec (with seq num) write speed all = 1.342979681 MB/sec (without seq num) 2024/11/23-11:10:48.655013 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257254: 919300 keys, 35793330 bytes, temperature: kWarm 2024/11/23-11:10:48.655086 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331448655049, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257254, "file_size": 35793330, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3280379165, "largest_seqno": 3921317649, "table_properties": {"data_size": 29496096, "index_size": 1760640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331445, "oldest_key_time": 0, "file_creation_time": 1732331445, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257254, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:10:48.692584 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257255.sst) /dev/shm/ToplingTemp/Topling-33512-7TPWRe 2024/11/23-11:10:49.751428 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257255.sst): first pass time = 1.06's, 126.764'MB/sec 2024/11/23-11:10:51.066894 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257255.sst): old prealloc_size = 147639500, real_size = 33428896 2024/11/23-11:10:51.067479 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257255.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933549 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0314 GB avg-val = 115.77 avg-zval = 33.66 2024/11/23-11:10:51.068308 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257255.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:10:51.919366 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257255.sst): second pass time = 1.31's, 82.210'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.183'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3154.984'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 293.703'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35296.282'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.210'MB/sec zip pipeline throughput = 82.210'MB/sec entries = 933549 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.77 avg-zval = 33.66 usrkeys = 933549 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.77 avg-zval = 33.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933549 bytes = 3384128 } seq expand size = 6534843 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5141 value = 3.4399 dict = 1.00 all = 3.5636 } Zip/UnZip{ index = 0.0951 value = 0.2907 dict = 1.00 all = 0.2806 } ---------------------------- total value len = 9.867964 GB avg = 0.162 KB (by entry num) total key len = 1.679350 GB avg = 0.028 KB total ukey len = 1.168256 GB avg = 0.016 KB total ukey num = 0.074069331 Billion total entry num = 0.060877060 Billion write speed all = 1.418640474 MB/sec (with seq num) write speed all = 1.358808269 MB/sec (without seq num) 2024/11/23-11:10:51.964519 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257255: 933549 keys, 37663924 bytes, temperature: kWarm 2024/11/23-11:10:51.964587 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331451964552, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257255, "file_size": 37663924, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3945178307, "table_properties": {"data_size": 31419536, "index_size": 1775808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26139372, "raw_average_key_size": 28, "raw_value_size": 108078498, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933549, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331448, "oldest_key_time": 0, "file_creation_time": 1732331448, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257255, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:10:52.003210 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257256.sst) /dev/shm/ToplingTemp/Topling-33512-th20Yf 2024/11/23-11:10:53.035245 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257256.sst): first pass time = 1.03's, 130.057'MB/sec 2024/11/23-11:10:54.323073 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257256.sst): old prealloc_size = 147639500, real_size = 30568256 2024/11/23-11:10:54.323620 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257256.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928550 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0286 GB avg-val = 116.55 avg-zval = 30.75 2024/11/23-11:10:54.325374 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257256.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:10:55.153099 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257256.sst): second pass time = 1.29's, 84.085'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.631'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3407.014'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 268.744'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33532.700'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.085'MB/sec zip pipeline throughput = 84.085'MB/sec entries = 928550 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.55 avg-zval = 30.75 usrkeys = 928550 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.55 avg-zval = 30.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928550 bytes = 3133872 } seq expand size = 6499850 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4416 value = 3.7895 dict = 1.00 all = 3.8843 } Zip/UnZip{ index = 0.0958 value = 0.2639 dict = 1.00 all = 0.2574 } ---------------------------- total value len = 9.982683 GB avg = 0.162 KB (by entry num) total key len = 1.705350 GB avg = 0.028 KB total ukey len = 1.186827 GB avg = 0.016 KB total ukey num = 0.074997881 Billion total entry num = 0.061805610 Billion write speed all = 1.435358050 MB/sec (with seq num) write speed all = 1.374637355 MB/sec (without seq num) 2024/11/23-11:10:55.195248 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257256: 928550 keys, 34553756 bytes, temperature: kWarm 2024/11/23-11:10:55.195324 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331455195281, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257256, "file_size": 34553756, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3829690065, "largest_seqno": 3945177528, "table_properties": {"data_size": 28557360, "index_size": 1778560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25999400, "raw_average_key_size": 28, "raw_value_size": 108218448, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928550, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331451, "oldest_key_time": 0, "file_creation_time": 1732331451, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257256, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:10:55.239472 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257257.sst) /dev/shm/ToplingTemp/Topling-33512-qz72Vd 2024/11/23-11:10:56.130905 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257257.sst): first pass time = 0.89's, 150.571'MB/sec 2024/11/23-11:10:57.416700 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257257.sst): old prealloc_size = 147639500, real_size = 32295200 2024/11/23-11:10:57.417266 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257257.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920997 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0303 GB avg-val = 117.73 avg-zval = 32.91 2024/11/23-11:10:57.419179 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257257.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:10:58.244740 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257257.sst): second pass time = 1.28's, 84.409'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.212'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3218.694'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 283.793'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 37579.944'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.409'MB/sec zip pipeline throughput = 84.409'MB/sec entries = 920997 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.73 avg-zval = 32.91 usrkeys = 920997 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.73 avg-zval = 32.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920997 bytes = 3453760 } seq expand size = 6446979 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5129 value = 3.5770 dict = 1.00 all = 3.6663 } Zip/UnZip{ index = 0.0951 value = 0.2796 dict = 1.00 all = 0.2728 } ---------------------------- total value len = 10.097559 GB avg = 0.161 KB (by entry num) total key len = 1.731137 GB avg = 0.028 KB total ukey len = 1.205247 GB avg = 0.016 KB total ukey num = 0.075918878 Billion total entry num = 0.062726607 Billion write speed all = 1.452081178 MB/sec (with seq num) write speed all = 1.390479041 MB/sec (without seq num) 2024/11/23-11:10:58.288961 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257257: 920997 keys, 36608328 bytes, temperature: kWarm 2024/11/23-11:10:58.289040 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331458289004, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257257, "file_size": 36608328, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3945174957, "table_properties": {"data_size": 30312656, "index_size": 1752128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25787916, "raw_average_key_size": 28, "raw_value_size": 108429814, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920997, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331455, "oldest_key_time": 0, "file_creation_time": 1732331455, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257257, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:10:58.333499 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257258.sst) /dev/shm/ToplingTemp/Topling-33512-QOcHWd 2024/11/23-11:10:59.390932 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257258.sst): first pass time = 1.06's, 126.934'MB/sec 2024/11/23-11:11:00.724183 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257258.sst): old prealloc_size = 147639500, real_size = 33696928 2024/11/23-11:11:00.724721 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257258.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934563 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0317 GB avg-val = 115.62 avg-zval = 33.89 2024/11/23-11:11:00.725553 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257258.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:11:01.577514 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257258.sst): second pass time = 1.33's, 81.093'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.593'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3484.555'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 295.979'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 27145.606'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.093'MB/sec zip pipeline throughput = 81.093'MB/sec entries = 934563 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.62 avg-zval = 33.89 usrkeys = 934563 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.62 avg-zval = 33.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934563 bytes = 3504624 } seq expand size = 6541941 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4431 value = 3.4114 dict = 1.00 all = 3.5273 } Zip/UnZip{ index = 0.0958 value = 0.2931 dict = 1.00 all = 0.2835 } ---------------------------- total value len = 10.212151 GB avg = 0.160 KB (by entry num) total key len = 1.757305 GB avg = 0.028 KB total ukey len = 1.223938 GB avg = 0.016 KB total ukey num = 0.076853441 Billion total entry num = 0.063661170 Billion write speed all = 1.468759818 MB/sec (with seq num) write speed all = 1.406265440 MB/sec (without seq num) 2024/11/23-11:11:01.626196 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257258: 934563 keys, 38051538 bytes, temperature: kWarm 2024/11/23-11:11:01.626267 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331461626230, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257258, "file_size": 38051538, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3945178395, "table_properties": {"data_size": 31673296, "index_size": 1789824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26167764, "raw_average_key_size": 28, "raw_value_size": 108050042, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934563, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331458, "oldest_key_time": 0, "file_creation_time": 1732331458, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257258, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:11:01.664333 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257259.sst) /dev/shm/ToplingTemp/Topling-33512-mIVyQc 2024/11/23-11:11:02.623462 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257259.sst): first pass time = 0.96's, 139.944'MB/sec 2024/11/23-11:11:03.932786 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257259.sst): old prealloc_size = 147639500, real_size = 33500832 2024/11/23-11:11:03.933379 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257259.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926822 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.82 avg-zval = 33.98 2024/11/23-11:11:03.934204 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257259.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:11:04.800551 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257259.sst): second pass time = 1.31's, 82.756'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.464'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3112.223'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 293.460'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35298.987'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.756'MB/sec zip pipeline throughput = 82.756'MB/sec entries = 926822 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.82 avg-zval = 33.98 usrkeys = 926822 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.82 avg-zval = 33.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926822 bytes = 3359744 } seq expand size = 6487754 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4447 value = 3.4377 dict = 1.00 all = 3.5587 } Zip/UnZip{ index = 0.0957 value = 0.2909 dict = 1.00 all = 0.2810 } ---------------------------- total value len = 10.326906 GB avg = 0.160 KB (by entry num) total key len = 1.783256 GB avg = 0.028 KB total ukey len = 1.242475 GB avg = 0.016 KB total ukey num = 0.077780263 Billion total entry num = 0.064587992 Billion write speed all = 1.485438160 MB/sec (with seq num) write speed all = 1.422059014 MB/sec (without seq num) 2024/11/23-11:11:04.845835 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257259: 926822 keys, 37715206 bytes, temperature: kWarm 2024/11/23-11:11:04.845901 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331464845868, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257259, "file_size": 37715206, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3945177591, "table_properties": {"data_size": 31494224, "index_size": 1774720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951016, "raw_average_key_size": 28, "raw_value_size": 108266716, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926822, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331461, "oldest_key_time": 0, "file_creation_time": 1732331461, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257259, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:11:04.883943 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257260.sst) /dev/shm/ToplingTemp/Topling-33512-SD9mbe 2024/11/23-11:11:05.894219 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257260.sst): first pass time = 1.01's, 132.859'MB/sec 2024/11/23-11:11:07.213321 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257260.sst): old prealloc_size = 147639500, real_size = 32735312 2024/11/23-11:11:07.213848 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257260.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930932 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.18 avg-zval = 33.01 2024/11/23-11:11:07.214690 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257260.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-11:11:08.073318 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257260.sst): second pass time = 1.32's, 82.071'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.979'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3517.584'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 287.539'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33778.140'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.071'MB/sec zip pipeline throughput = 82.071'MB/sec entries = 930932 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.18 avg-zval = 33.01 usrkeys = 930932 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.18 avg-zval = 33.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930932 bytes = 3374640 } seq expand size = 6516524 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5126 value = 3.5193 dict = 1.00 all = 3.6313 } Zip/UnZip{ index = 0.0951 value = 0.2842 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 10.441574 GB avg = 0.159 KB (by entry num) total key len = 1.809322 GB avg = 0.028 KB total ukey len = 1.261093 GB avg = 0.016 KB total ukey num = 0.078711195 Billion total entry num = 0.065518924 Billion write speed all = 1.502097691 MB/sec (with seq num) write speed all = 1.437830835 MB/sec (without seq num) 2024/11/23-11:11:08.119662 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257260: 930932 keys, 36961256 bytes, temperature: kWarm 2024/11/23-11:11:08.119734 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331468119696, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257260, "file_size": 36961256, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3945178272, "table_properties": {"data_size": 30731328, "index_size": 1771072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26066096, "raw_average_key_size": 28, "raw_value_size": 108151640, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930932, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331464, "oldest_key_time": 0, "file_creation_time": 1732331464, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257260, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:11:08.158772 139957290698304 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257261.sst) /dev/shm/ToplingTemp/Topling-33512-5uaatb 2024/11/23-11:11:08.331934 139957290698304 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257261.sst): first pass time = 0.17's, 233.074'MB/sec 2024/11/23-11:11:08.716913 139957290698304 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257261.sst): old prealloc_size = 147639500, real_size = 10424032 2024/11/23-11:11:08.717166 139957290698304 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257261.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 283482 prefix = 4 raw-key = 0.0045 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0324 GB zip-val = 0.0098 GB avg-val = 114.34 avg-zval = 34.60 2024/11/23-11:11:08.717487 139957290698304 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257261.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0007 GB, waited 0.000 sec, Key+Value = 0.040 GB 2024/11/23-11:11:08.968119 139957290698304 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00/257261.sst): second pass time = 0.38's, 84.356'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 46.631'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2387.209'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.335'MB/sec write SST data time = 0.04's, 296.924'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.330'MB, throughput = 8637.088'MB/sec zip my value time = 0.38's, unzip length = 0.032'GB zip my value throughput = 84.356'MB/sec zip pipeline throughput = 84.356'MB/sec entries = 283482 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.34 avg-zval = 34.60 usrkeys = 283482 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.34 avg-zval = 34.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 283482 bytes = 1027632 } seq expand size = 1984374 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0057 GB value = 0.0324 GB dict = 0.33 MB all = 0.0404 GB } __ZipSize{ index = 0.0005 GB value = 0.0098 GB dict = 0.33 MB all = 0.0117 GB } UnZip/Zip{ index = 10.4209 value = 3.3044 dict = 1.00 all = 3.4449 } Zip/UnZip{ index = 0.0960 value = 0.3026 dict = 1.00 all = 0.2903 } ---------------------------- total value len = 10.475971 GB avg = 0.159 KB (by entry num) total key len = 1.817260 GB avg = 0.028 KB total ukey len = 1.266763 GB avg = 0.016 KB total ukey num = 0.078994677 Billion total entry num = 0.065802406 Billion write speed all = 1.507123013 MB/sec (with seq num) write speed all = 1.442585174 MB/sec (without seq num) 2024/11/23-11:11:08.988578 139957290698304 [compaction_job.cc:1948] [default] [JOB 24943] Generated table #257261: 283482 keys, 11712949 bytes, temperature: kWarm 2024/11/23-11:11:08.988636 139957290698304 EVENT_LOG_v1 {"time_micros": 1732331468988606, "cf_name": "default", "job": 24943, "event": "table_file_creation", "file_number": 257261, "file_size": 11712949, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3945178164, "table_properties": {"data_size": 9808912, "index_size": 544064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7937496, "raw_average_key_size": 28, "raw_value_size": 32412544, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 283482, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732331468, "oldest_key_time": 0, "file_creation_time": 1732331468, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 257261, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-11:11:09.004300 139957290698304 [compaction_job.cc:703] job-24943: subcompact[0], size: 0.523744 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-24943/att-00 [257247,257248,257249,257250,257251,257252,257253,257254,257255,257256,257257,257258,257259,257260,257261] 2024/11/23-11:11:09.091121 139957290698304 (Original Log Time 2024/11/23-11:10:22.582766) EVENT_LOG_v1 {"time_micros": 1732331422582726, "job": 24943, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [257175], "files_L3": [257233, 257116, 257136, 257137, 257138, 257139, 257140, 257141, 257142, 257143, 257144, 257145, 257146, 257147, 257234], "score": 0.998357, "input_data_size": 546405375, "oldest_snapshot_seqno": 3945495239} 2024/11/23-11:11:09.091125 139957290698304 (Original Log Time 2024/11/23-11:11:09.089815) [compaction_job.cc:2022] [default] [JOB 24943] Compacted 1@2 + 15@3 files to L3 => 523744474 bytes 2024/11/23-11:11:09.091126 139957290698304 (Original Log Time 2024/11/23-11:11:09.091056) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.8 rd, 11.3 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(27.1, 494.0 +0.0 blob) out(499.5 +0.0 blob), read-write-amplify(37.6) write-amplify(18.4) OK, records in: 13511741, records dropped: 67499 output_compression: Snappy 2024/11/23-11:11:09.091132 139957290698304 (Original Log Time 2024/11/23-11:11:09.091101) EVENT_LOG_v1 {"time_micros": 1732331469091081, "cf": "default", "job": 24943, "event": "compaction_finished", "compaction_time_micros": 46421541, "compaction_time_cpu_micros": 19774793, "output_level": 3, "num_output_files": 15, "total_output_size": 523744474, "num_input_records": 13511741, "num_output_records": 13444242, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 357485515, "file_range_sync_nanos": 0, "file_fsync_nanos": 9984, "file_prepare_write_nanos": 44684, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-11:11:09.235624 139957290698304 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732331422,"executesMs":46652,"compactionJobId":24943,"attempt":0,"compactionInputRawBytes":1928983979,"compactionInputZipBytes":546405375,"compactionOutputRawBytes":0,"compactionOutputZipBytes":523744474,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-e41f79e1e16ce84fd2891b49f9c5a4e6-1f6501e4f9911fa9-00"}}, response = 2024/11/23-11:11:09.235705 139957290698304 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-24943/att-00: olev 3, work 46.653 s, result 7.720 ms, install 1.339 ms, input{raw 1.797 GiB zip 521.093 MiB}