2024/11/23-15:04:38.088428 139957685155392 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00: fork to child time = 0.007963 sec 2024/11/23-15:04:38.089170 139957685155392 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00 2024/11/23-15:04:38.096175 139957685155392 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00 2024/11/23-15:04:38.097071 139957685155392 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-15:04:38.097279 139957685155392 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-15:04:38.152178 139957685155392 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28716/att-00/MANIFEST-206347 2024/11/23-15:04:38.182319 139957685155392 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28716/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 265276, last_sequence is 4051232735, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 265154 2024/11/23-15:04:38.182325 139957685155392 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-15:04:38.182436 139957685155392 [version_set.cc:5721] Creating manifest 265276 2024/11/23-15:04:38.218566 139957685155392 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-15:04:38.243452 139957685155392 [compaction_job.cc:2437] [default] [JOB 28716] Compacting 1@2 + 20@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-15:04:38.243469 139957685155392 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 21458 Base level 2, inputs: [265230(31MB)], [265157(40MB) 265158(40MB) 265159(40MB) 265160(40MB) 265161(40MB) 265162(40MB) 265163(40MB) 265164(40MB) 265165(40MB) 265166(40MB) 265167(40MB) 265168(40MB) 265169(40MB) 265170(40MB) 265171(40MB) 265172(40MB) 265173(40MB) 265174(40MB) 265175(39MB) 265272(40MB)] 2024/11/23-15:04:38.251772 139957685155392 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:04:38.251795 139957685155392 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:04:38.251816 139957685155392 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:04:38.251825 139957685155392 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:04:38.251836 139957685155392 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:04:38.885191 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265277.sst) /dev/shm/ToplingTemp/Topling-86185-ldtYL6 2024/11/23-15:04:39.079853 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265277.sst): first pass time = 0.19's, 689.658'MB/sec 2024/11/23-15:04:40.055896 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265277.sst): old prealloc_size = 147639500, real_size = 40674656 2024/11/23-15:04:40.056045 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265277.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.99 avg-zval = 312.20 2024/11/23-15:04:40.056218 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265277.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:04:40.226377 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265277.sst): second pass time = 0.98's, 134.978'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 249.353'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1893.473'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.05's, 866.069'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35239.751'MB/sec zip my value time = 0.98's, unzip length = 0.132'GB zip my value throughput = 134.978'MB/sec zip pipeline throughput = 159.540'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.99 avg-zval = 312.20 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.99 avg-zval = 312.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6470 value = 3.2575 dict = 1.00 all = 3.1575 } Zip/UnZip{ index = 0.1504 value = 0.3070 dict = 1.00 all = 0.3167 } ---------------------------- total value len = 8.981155 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679865 Billion total entry num = 0.052487594 Billion write speed all = 0.470252581 MB/sec (with seq num) write speed all = 0.451310839 MB/sec (without seq num) 2024/11/23-15:04:40.297318 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265277: 129430 keys, 42507387 bytes, temperature: kWarm 2024/11/23-15:04:40.297382 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345480297340, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265277, "file_size": 42507387, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4050146896, "table_properties": {"data_size": 40408464, "index_size": 233664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629247, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345478, "oldest_key_time": 0, "file_creation_time": 1732345478, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265277, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:04:40.338245 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265278.sst) /dev/shm/ToplingTemp/Topling-86185-McyJt8 2024/11/23-15:04:41.005393 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265278.sst): first pass time = 0.67's, 201.190'MB/sec 2024/11/23-15:04:41.869434 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265278.sst): old prealloc_size = 147639500, real_size = 40829216 2024/11/23-15:04:41.869557 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265278.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.98 avg-zval = 313.44 2024/11/23-15:04:41.869742 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265278.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:04:42.058864 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265278.sst): second pass time = 0.86's, 152.454'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 225.233'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2189.401'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.241'MB/sec write SST data time = 0.06's, 771.204'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45340.901'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 152.454'MB/sec zip pipeline throughput = 152.454'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1016.98 avg-zval = 313.44 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1023.98 avg-zval = 313.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8169 value = 3.2446 dict = 1.00 all = 3.1461 } Zip/UnZip{ index = 0.1467 value = 0.3082 dict = 1.00 all = 0.3179 } ---------------------------- total value len = 9.113690 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809296 Billion total entry num = 0.052617025 Billion write speed all = 0.476308648 MB/sec (with seq num) write speed all = 0.457321766 MB/sec (without seq num) 2024/11/23-15:04:42.113208 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265278: 129431 keys, 42662013 bytes, temperature: kWarm 2024/11/23-15:04:42.113296 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345482113250, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265278, "file_size": 42662013, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4050133148, "table_properties": {"data_size": 40568848, "index_size": 227840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629152, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345480, "oldest_key_time": 0, "file_creation_time": 1732345480, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265278, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:04:42.163352 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265279.sst) /dev/shm/ToplingTemp/Topling-86185-6Vjgt8 2024/11/23-15:04:42.740187 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265279.sst): first pass time = 0.58's, 232.700'MB/sec 2024/11/23-15:04:45.326813 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265279.sst): old prealloc_size = 147639500, real_size = 40679424 2024/11/23-15:04:45.326962 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265279.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.97 avg-zval = 312.24 2024/11/23-15:04:45.327165 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265279.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:04:45.717580 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265279.sst): second pass time = 2.59's, 50.909'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.39's, 108.798'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1704.964'MB/sec (index lex order gen) rebuild zvType time = 0.26's, 0.125'MB/sec write SST data time = 0.13's, 321.913'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45672.137'MB/sec zip my value time = 2.59's, unzip length = 0.132'GB zip my value throughput = 50.909'MB/sec zip pipeline throughput = 107.430'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.97 avg-zval = 312.24 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.97 avg-zval = 312.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6581 value = 3.2571 dict = 1.00 all = 3.1572 } Zip/UnZip{ index = 0.1502 value = 0.3070 dict = 1.00 all = 0.3167 } ---------------------------- total value len = 9.246226 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938729 Billion total entry num = 0.052746458 Billion write speed all = 0.482324028 MB/sec (with seq num) write speed all = 0.463293582 MB/sec (without seq num) 2024/11/23-15:04:45.787098 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265279: 129433 keys, 42511907 bytes, temperature: kWarm 2024/11/23-15:04:45.787165 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345485787129, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265279, "file_size": 42511907, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237870807, "largest_seqno": 4050142432, "table_properties": {"data_size": 40413616, "index_size": 233280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131630033, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345482, "oldest_key_time": 0, "file_creation_time": 1732345482, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265279, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:04:45.826019 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265280.sst) /dev/shm/ToplingTemp/Topling-86185-zZLqs7 2024/11/23-15:04:46.394125 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265280.sst): first pass time = 0.57's, 236.273'MB/sec 2024/11/23-15:04:47.328655 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265280.sst): old prealloc_size = 147639500, real_size = 40817232 2024/11/23-15:04:47.328792 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265280.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.95 avg-zval = 313.34 2024/11/23-15:04:47.328997 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265280.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:04:47.514954 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265280.sst): second pass time = 0.93's, 140.970'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 228.929'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1873.481'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.242'MB/sec write SST data time = 0.05's, 811.838'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46383.966'MB/sec zip my value time = 0.93's, unzip length = 0.132'GB zip my value throughput = 140.970'MB/sec zip pipeline throughput = 140.970'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1016.95 avg-zval = 313.34 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1023.95 avg-zval = 313.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129435 bytes = 485392 } seq expand size = 906045 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8076 value = 3.2456 dict = 1.00 all = 3.1470 } Zip/UnZip{ index = 0.1469 value = 0.3081 dict = 1.00 all = 0.3178 } ---------------------------- total value len = 9.378762 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068164 Billion total entry num = 0.052875893 Billion write speed all = 0.488378383 MB/sec (with seq num) write speed all = 0.469302784 MB/sec (without seq num) 2024/11/23-15:04:47.568136 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265280: 129435 keys, 42650113 bytes, temperature: kWarm 2024/11/23-15:04:47.568197 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345487568165, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265280, "file_size": 42650113, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4050136307, "table_properties": {"data_size": 40556544, "index_size": 228160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629320, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345485, "oldest_key_time": 0, "file_creation_time": 1732345485, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265280, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:04:47.612437 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265281.sst) /dev/shm/ToplingTemp/Topling-86185-u6kc96 2024/11/23-15:04:48.496074 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265281.sst): first pass time = 0.88's, 151.900'MB/sec 2024/11/23-15:04:50.363011 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265281.sst): old prealloc_size = 147639500, real_size = 40593712 2024/11/23-15:04:50.363162 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265281.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.56 2024/11/23-15:04:50.363351 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265281.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:04:50.688406 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265281.sst): second pass time = 1.87's, 70.537'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.33's, 130.383'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1897.628'MB/sec (index lex order gen) rebuild zvType time = 0.26's, 0.123'MB/sec write SST data time = 0.06's, 691.419'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 35607.024'MB/sec zip my value time = 1.87's, unzip length = 0.132'GB zip my value throughput = 70.537'MB/sec zip pipeline throughput = 133.594'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1017.00 avg-zval = 311.56 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1024.00 avg-zval = 311.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5892 value = 3.2642 dict = 1.00 all = 3.1636 } Zip/UnZip{ index = 0.1518 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.511297 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197593 Billion total entry num = 0.053005322 Billion write speed all = 0.494401084 MB/sec (with seq num) write speed all = 0.475281527 MB/sec (without seq num) 2024/11/23-15:04:50.743668 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265281: 129429 keys, 42426396 bytes, temperature: kWarm 2024/11/23-15:04:50.743749 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345490743702, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265281, "file_size": 42426396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4050143353, "table_properties": {"data_size": 40325472, "index_size": 235712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629570, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345487, "oldest_key_time": 0, "file_creation_time": 1732345487, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265281, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:04:50.805417 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265282.sst) /dev/shm/ToplingTemp/Topling-86185-aoyXc8 2024/11/23-15:04:51.353256 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265282.sst): first pass time = 0.55's, 245.011'MB/sec 2024/11/23-15:04:52.227803 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265282.sst): old prealloc_size = 147639500, real_size = 40889600 2024/11/23-15:04:52.227939 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265282.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1017.01 avg-zval = 313.90 2024/11/23-15:04:52.228111 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265282.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:04:52.401266 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265282.sst): second pass time = 0.87's, 150.630'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 246.314'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2177.442'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.05's, 797.432'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47463.706'MB/sec zip my value time = 0.87's, unzip length = 0.132'GB zip my value throughput = 150.630'MB/sec zip pipeline throughput = 150.630'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1017.01 avg-zval = 313.90 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1024.01 avg-zval = 313.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129428 bytes = 485376 } seq expand size = 905996 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.7806 value = 3.2399 dict = 1.00 all = 3.1416 } Zip/UnZip{ index = 0.1475 value = 0.3087 dict = 1.00 all = 0.3183 } ---------------------------- total value len = 9.643832 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327021 Billion total entry num = 0.053134750 Billion write speed all = 0.500454999 MB/sec (with seq num) write speed all = 0.481290236 MB/sec (without seq num) 2024/11/23-15:04:52.453343 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265282: 129428 keys, 42722112 bytes, temperature: kWarm 2024/11/23-15:04:52.453409 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345492453373, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265282, "file_size": 42722112, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237885133, "largest_seqno": 4050135251, "table_properties": {"data_size": 40628016, "index_size": 229056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629246, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345490, "oldest_key_time": 0, "file_creation_time": 1732345490, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265282, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:04:52.509630 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265283.sst) /dev/shm/ToplingTemp/Topling-86185-iOP4X7 2024/11/23-15:04:53.353733 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265283.sst): first pass time = 0.84's, 159.017'MB/sec 2024/11/23-15:04:54.572666 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265283.sst): old prealloc_size = 147639500, real_size = 41009552 2024/11/23-15:04:54.572783 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265283.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.02 avg-zval = 314.87 2024/11/23-15:04:54.572937 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265283.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:04:54.736742 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265283.sst): second pass time = 1.21's, 108.861'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.136'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2402.292'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 939.091'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47361.746'MB/sec zip my value time = 1.21's, unzip length = 0.132'GB zip my value throughput = 108.861'MB/sec zip pipeline throughput = 125.852'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.02 avg-zval = 314.87 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.02 avg-zval = 314.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129427 bytes = 485360 } seq expand size = 905989 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9376 value = 3.2299 dict = 1.00 all = 3.1329 } Zip/UnZip{ index = 0.1441 value = 0.3096 dict = 1.00 all = 0.3192 } ---------------------------- total value len = 9.776369 GB avg = 0.184 KB (by entry num) total key len = 1.458941 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456448 Billion total entry num = 0.053264177 Billion write speed all = 0.506493804 MB/sec (with seq num) write speed all = 0.487284383 MB/sec (without seq num) 2024/11/23-15:04:54.823105 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265283: 129427 keys, 42842245 bytes, temperature: kWarm 2024/11/23-15:04:54.823174 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345494823138, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265283, "file_size": 42842245, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4050146070, "table_properties": {"data_size": 40753152, "index_size": 223872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630215, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345492, "oldest_key_time": 0, "file_creation_time": 1732345492, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265283, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:04:54.861547 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265284.sst) /dev/shm/ToplingTemp/Topling-86185-DcgUG4 2024/11/23-15:04:55.395672 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265284.sst): first pass time = 0.53's, 251.304'MB/sec 2024/11/23-15:04:56.597043 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265284.sst): old prealloc_size = 147639500, real_size = 40947696 2024/11/23-15:04:56.597165 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265284.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1016.96 avg-zval = 314.40 2024/11/23-15:04:56.597330 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265284.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:04:56.761147 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265284.sst): second pass time = 1.20's, 109.718'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.723'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2294.723'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 942.117'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45827.913'MB/sec zip my value time = 1.20's, unzip length = 0.132'GB zip my value throughput = 109.718'MB/sec zip pipeline throughput = 135.373'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.96 avg-zval = 314.40 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.96 avg-zval = 314.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129435 bytes = 485392 } seq expand size = 906045 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 7.0203 value = 3.2346 dict = 1.00 all = 3.1374 } Zip/UnZip{ index = 0.1424 value = 0.3092 dict = 1.00 all = 0.3187 } ---------------------------- total value len = 9.908904 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585883 Billion total entry num = 0.053393612 Billion write speed all = 0.512538513 MB/sec (with seq num) write speed all = 0.493284169 MB/sec (without seq num) 2024/11/23-15:04:56.812397 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265284: 129435 keys, 42780632 bytes, temperature: kWarm 2024/11/23-15:04:56.812458 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345496812423, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265284, "file_size": 42780632, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874784, "largest_seqno": 4050148696, "table_properties": {"data_size": 40693920, "index_size": 221248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629719, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345494, "oldest_key_time": 0, "file_creation_time": 1732345494, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265284, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:04:56.849408 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265285.sst) /dev/shm/ToplingTemp/Topling-86185-BakIV6 2024/11/23-15:04:57.477010 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265285.sst): first pass time = 0.63's, 213.874'MB/sec 2024/11/23-15:04:58.257188 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265285.sst): old prealloc_size = 147639500, real_size = 40591968 2024/11/23-15:04:58.257314 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265285.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.55 2024/11/23-15:04:58.257474 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265285.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:04:58.421774 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265285.sst): second pass time = 0.78's, 168.883'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 257.815'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2271.744'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.05's, 910.878'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45150.031'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 168.883'MB/sec zip pipeline throughput = 168.883'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1016.99 avg-zval = 311.55 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1023.99 avg-zval = 311.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5911 value = 3.2643 dict = 1.00 all = 3.1637 } Zip/UnZip{ index = 0.1517 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.041440 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715314 Billion total entry num = 0.053523043 Billion write speed all = 0.518590628 MB/sec (with seq num) write speed all = 0.499291054 MB/sec (without seq num) 2024/11/23-15:04:58.472407 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265285: 129431 keys, 42425014 bytes, temperature: kWarm 2024/11/23-15:04:58.472477 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345498472442, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265285, "file_size": 42425014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873877, "largest_seqno": 4050142737, "table_properties": {"data_size": 40323792, "index_size": 235648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629965, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345496, "oldest_key_time": 0, "file_creation_time": 1732345496, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265285, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:04:58.511010 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265286.sst) /dev/shm/ToplingTemp/Topling-86185-gNTdD8 2024/11/23-15:04:59.012563 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265286.sst): first pass time = 0.50's, 267.630'MB/sec 2024/11/23-15:04:59.797769 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265286.sst): old prealloc_size = 147639500, real_size = 40592448 2024/11/23-15:04:59.797895 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265286.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.55 2024/11/23-15:04:59.798079 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265286.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:04:59.961936 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265286.sst): second pass time = 0.78's, 167.995'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.461'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2280.584'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 931.731'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42913.884'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 167.995'MB/sec zip pipeline throughput = 167.995'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1016.99 avg-zval = 311.55 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1023.99 avg-zval = 311.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5893 value = 3.2643 dict = 1.00 all = 3.1637 } Zip/UnZip{ index = 0.1518 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.173977 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844745 Billion total entry num = 0.053652474 Billion write speed all = 0.524644690 MB/sec (with seq num) write speed all = 0.505299789 MB/sec (without seq num) 2024/11/23-15:05:00.013882 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265286: 129431 keys, 42425110 bytes, temperature: kWarm 2024/11/23-15:05:00.013951 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345500013917, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265286, "file_size": 42425110, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4050145522, "table_properties": {"data_size": 40324208, "index_size": 235712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131630094, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345498, "oldest_key_time": 0, "file_creation_time": 1732345498, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265286, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:00.059819 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265287.sst) /dev/shm/ToplingTemp/Topling-86185-OVpQT6 2024/11/23-15:05:00.790233 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265287.sst): first pass time = 0.73's, 183.769'MB/sec 2024/11/23-15:05:01.606611 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265287.sst): old prealloc_size = 147639500, real_size = 40989200 2024/11/23-15:05:01.606746 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265287.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1016.98 avg-zval = 314.69 2024/11/23-15:05:01.606905 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265287.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:05:01.766005 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265287.sst): second pass time = 0.81's, 161.552'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 268.727'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2237.647'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.280'MB/sec write SST data time = 0.04's, 977.227'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44005.078'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 161.552'MB/sec zip pipeline throughput = 161.552'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1016.98 avg-zval = 314.69 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1023.98 avg-zval = 314.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.8690 value = 3.2317 dict = 1.00 all = 3.1343 } Zip/UnZip{ index = 0.1456 value = 0.3094 dict = 1.00 all = 0.3190 } ---------------------------- total value len = 10.306512 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974176 Billion total entry num = 0.053781905 Billion write speed all = 0.530691562 MB/sec (with seq num) write speed all = 0.511301569 MB/sec (without seq num) 2024/11/23-15:05:01.817032 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265287: 129431 keys, 42821722 bytes, temperature: kWarm 2024/11/23-15:05:01.817102 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345501817066, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265287, "file_size": 42821722, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237888398, "largest_seqno": 4050142980, "table_properties": {"data_size": 40730560, "index_size": 226112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629219, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345500, "oldest_key_time": 0, "file_creation_time": 1732345500, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265287, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:01.855301 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265288.sst) /dev/shm/ToplingTemp/Topling-86185-3Qluh6 2024/11/23-15:05:02.585557 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265288.sst): first pass time = 0.73's, 183.808'MB/sec 2024/11/23-15:05:03.344384 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265288.sst): old prealloc_size = 147639500, real_size = 40751936 2024/11/23-15:05:03.344506 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265288.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129423 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.05 avg-zval = 312.83 2024/11/23-15:05:03.344679 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265288.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:05:03.503589 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265288.sst): second pass time = 0.76's, 173.627'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.515'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2308.384'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 1018.823'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44218.489'MB/sec zip my value time = 0.76's, unzip length = 0.132'GB zip my value throughput = 173.627'MB/sec zip pipeline throughput = 173.627'MB/sec entries = 129423 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.05 avg-zval = 312.83 usrkeys = 129423 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.05 avg-zval = 312.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129423 bytes = 485344 } seq expand size = 905961 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.6796 value = 3.2512 dict = 1.00 all = 3.1518 } Zip/UnZip{ index = 0.1497 value = 0.3076 dict = 1.00 all = 0.3173 } ---------------------------- total value len = 10.439047 GB avg = 0.194 KB (by entry num) total key len = 1.471884 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103599 Billion total entry num = 0.053911328 Billion write speed all = 0.536739054 MB/sec (with seq num) write speed all = 0.517303923 MB/sec (without seq num) 2024/11/23-15:05:03.554587 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265288: 129423 keys, 42584680 bytes, temperature: kWarm 2024/11/23-15:05:03.554666 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345503554621, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265288, "file_size": 42584680, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237871365, "largest_seqno": 4050139139, "table_properties": {"data_size": 40486896, "index_size": 232512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588460, "raw_average_key_size": 20, "raw_value_size": 131629362, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129423, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345501, "oldest_key_time": 0, "file_creation_time": 1732345501, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265288, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:03.593333 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265289.sst) /dev/shm/ToplingTemp/Topling-86185-Ac6yH8 2024/11/23-15:05:04.167873 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265289.sst): first pass time = 0.57's, 233.626'MB/sec 2024/11/23-15:05:04.925722 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265289.sst): old prealloc_size = 147639500, real_size = 40496112 2024/11/23-15:05:04.925848 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265289.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.90 2024/11/23-15:05:04.925952 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265289.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:05:05.085265 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265289.sst): second pass time = 0.76's, 173.860'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.444'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2199.803'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 1007.030'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43862.572'MB/sec zip my value time = 0.76's, unzip length = 0.132'GB zip my value throughput = 173.860'MB/sec zip pipeline throughput = 173.860'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.00 avg-zval = 310.90 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1019.31 avg-zval = 310.90 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 42795 bytes = 160496 } seq expand size = 299565 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9397 value = 3.2711 dict = 1.00 all = 3.1938 } Zip/UnZip{ index = 0.1441 value = 0.3057 dict = 1.00 all = 0.3131 } ---------------------------- total value len = 10.570977 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233029 Billion total entry num = 0.054040758 Billion write speed all = 0.542762131 MB/sec (with seq num) write speed all = 0.523281728 MB/sec (without seq num) 2024/11/23-15:05:05.137632 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265289: 129430 keys, 42024497 bytes, temperature: kWarm 2024/11/23-15:05:05.137713 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345505137674, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265289, "file_size": 42024497, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4050143354, "table_properties": {"data_size": 40239776, "index_size": 223808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131630143, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345503, "oldest_key_time": 0, "file_creation_time": 1732345503, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265289, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:05.176306 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265290.sst) /dev/shm/ToplingTemp/Topling-86185-a9Hi67 2024/11/23-15:05:05.823722 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265290.sst): first pass time = 0.65's, 207.327'MB/sec 2024/11/23-15:05:06.641492 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265290.sst): old prealloc_size = 147639500, real_size = 41050272 2024/11/23-15:05:06.641615 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265290.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.98 avg-zval = 315.20 2024/11/23-15:05:06.641782 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265290.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:05:06.801750 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265290.sst): second pass time = 0.82's, 161.112'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.364'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2161.002'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 1003.554'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45616.289'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 161.112'MB/sec zip pipeline throughput = 161.112'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1016.98 avg-zval = 315.20 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1023.09 avg-zval = 315.20 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 112955 bytes = 423600 } seq expand size = 790685 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 7.0405 value = 3.2265 dict = 1.00 all = 3.1329 } Zip/UnZip{ index = 0.1420 value = 0.3099 dict = 1.00 all = 0.3192 } ---------------------------- total value len = 10.703397 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362461 Billion total entry num = 0.054170190 Billion write speed all = 0.548803140 MB/sec (with seq num) write speed all = 0.529277590 MB/sec (without seq num) 2024/11/23-15:05:06.853119 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265290: 129432 keys, 42841391 bytes, temperature: kWarm 2024/11/23-15:05:06.853190 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345506853153, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265290, "file_size": 42841391, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4050123328, "table_properties": {"data_size": 40797136, "index_size": 220608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131630019, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345505, "oldest_key_time": 0, "file_creation_time": 1732345505, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265290, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:06.889815 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265291.sst) /dev/shm/ToplingTemp/Topling-86185-GkciY5 2024/11/23-15:05:07.440662 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265291.sst): first pass time = 0.55's, 243.672'MB/sec 2024/11/23-15:05:08.196062 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265291.sst): old prealloc_size = 147639500, real_size = 40682192 2024/11/23-15:05:08.196180 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265291.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.00 avg-zval = 312.32 2024/11/23-15:05:08.196308 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265291.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:05:08.354809 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265291.sst): second pass time = 0.75's, 174.594'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.763'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2321.233'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.04's, 1001.298'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 37309.085'MB/sec zip my value time = 0.75's, unzip length = 0.132'GB zip my value throughput = 174.594'MB/sec zip pipeline throughput = 174.594'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1017.00 avg-zval = 312.32 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1021.22 avg-zval = 312.32 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 77903 bytes = 292144 } seq expand size = 545321 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0423 GB } UnZip/Zip{ index = 6.8631 value = 3.2563 dict = 1.00 all = 3.1699 } Zip/UnZip{ index = 0.1457 value = 0.3071 dict = 1.00 all = 0.3155 } ---------------------------- total value len = 10.835573 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021175 GB avg = 0.015 KB total ukey num = 0.067491890 Billion total entry num = 0.054299619 Billion write speed all = 0.554836234 MB/sec (with seq num) write speed all = 0.535265401 MB/sec (without seq num) 2024/11/23-15:05:08.405191 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265291: 129429 keys, 42342154 bytes, temperature: kWarm 2024/11/23-15:05:08.405262 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345508405225, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265291, "file_size": 42342154, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4050135809, "table_properties": {"data_size": 40423360, "index_size": 226304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629824, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345506, "oldest_key_time": 0, "file_creation_time": 1732345506, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265291, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:08.445792 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265292.sst) /dev/shm/ToplingTemp/Topling-86185-wpOg06 2024/11/23-15:05:09.172946 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265292.sst): first pass time = 0.73's, 184.591'MB/sec 2024/11/23-15:05:09.961311 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265292.sst): old prealloc_size = 147639500, real_size = 40985760 2024/11/23-15:05:09.961430 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265292.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.04 avg-zval = 314.68 2024/11/23-15:05:09.961594 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265292.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:05:10.120581 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265292.sst): second pass time = 0.79's, 167.308'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 268.817'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2309.951'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.04's, 997.256'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49280.602'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 167.308'MB/sec zip pipeline throughput = 167.308'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1017.04 avg-zval = 314.68 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.59 avg-zval = 314.68 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 121111 bytes = 454176 } seq expand size = 847777 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.8882 value = 3.2319 dict = 1.00 all = 3.1353 } Zip/UnZip{ index = 0.1452 value = 0.3094 dict = 1.00 all = 0.3189 } ---------------------------- total value len = 10.968050 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621315 Billion total entry num = 0.054429044 Billion write speed all = 0.560876743 MB/sec (with seq num) write speed all = 0.541260822 MB/sec (without seq num) 2024/11/23-15:05:10.171083 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265292: 129425 keys, 42808236 bytes, temperature: kWarm 2024/11/23-15:05:10.171153 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345510171119, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265292, "file_size": 42808236, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4050135402, "table_properties": {"data_size": 40727760, "index_size": 225472, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131630073, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345508, "oldest_key_time": 0, "file_creation_time": 1732345508, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265292, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:10.228870 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265293.sst) /dev/shm/ToplingTemp/Topling-86185-8vOiH7 2024/11/23-15:05:10.852126 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265293.sst): first pass time = 0.62's, 215.368'MB/sec 2024/11/23-15:05:11.603578 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265293.sst): old prealloc_size = 147639500, real_size = 40513776 2024/11/23-15:05:11.603713 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265293.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.02 2024/11/23-15:05:11.603818 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265293.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:05:11.766541 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265293.sst): second pass time = 0.75's, 175.334'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.188'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1980.106'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.04's, 1001.008'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47071.185'MB/sec zip my value time = 0.75's, unzip length = 0.132'GB zip my value throughput = 175.334'MB/sec zip pipeline throughput = 175.334'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.99 avg-zval = 311.02 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1019.71 avg-zval = 311.02 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 50291 bytes = 188608 } seq expand size = 352037 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.8866 value = 3.2698 dict = 1.00 all = 3.1903 } Zip/UnZip{ index = 0.1452 value = 0.3058 dict = 1.00 all = 0.3134 } ---------------------------- total value len = 11.100032 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750746 Billion total entry num = 0.054558475 Billion write speed all = 0.566896998 MB/sec (with seq num) write speed all = 0.547235889 MB/sec (without seq num) 2024/11/23-15:05:11.816492 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265293: 129431 keys, 42070151 bytes, temperature: kWarm 2024/11/23-15:05:11.816563 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345511816527, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265293, "file_size": 42070151, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4050135426, "table_properties": {"data_size": 40255712, "index_size": 225536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629488, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345510, "oldest_key_time": 0, "file_creation_time": 1732345510, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265293, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:11.852777 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265294.sst) /dev/shm/ToplingTemp/Topling-86185-6BiVz7 2024/11/23-15:05:12.346706 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265294.sst): first pass time = 0.49's, 271.757'MB/sec 2024/11/23-15:05:13.103778 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265294.sst): old prealloc_size = 147639500, real_size = 40585680 2024/11/23-15:05:13.103898 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265294.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.03 avg-zval = 311.61 2024/11/23-15:05:13.104001 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265294.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:05:13.259072 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265294.sst): second pass time = 0.76's, 174.030'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 270.832'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2268.530'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.282'MB/sec write SST data time = 0.04's, 1040.750'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48958.282'MB/sec zip my value time = 0.76's, unzip length = 0.132'GB zip my value throughput = 174.030'MB/sec zip pipeline throughput = 174.030'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.03 avg-zval = 311.61 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.48 avg-zval = 311.61 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 26879 bytes = 100816 } seq expand size = 188153 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9935 value = 3.2637 dict = 1.00 all = 3.1916 } Zip/UnZip{ index = 0.1430 value = 0.3064 dict = 1.00 all = 0.3133 } ---------------------------- total value len = 11.231850 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880172 Billion total entry num = 0.054687901 Billion write speed all = 0.572912962 MB/sec (with seq num) write speed all = 0.553206538 MB/sec (without seq num) 2024/11/23-15:05:13.310415 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265294: 129426 keys, 42053894 bytes, temperature: kWarm 2024/11/23-15:05:13.310486 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345513310450, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265294, "file_size": 42053894, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4050130449, "table_properties": {"data_size": 40331072, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131630140, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345511, "oldest_key_time": 0, "file_creation_time": 1732345511, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265294, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:13.356648 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265295.sst) /dev/shm/ToplingTemp/Topling-86185-4dkwE7 2024/11/23-15:05:14.066851 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265295.sst): first pass time = 0.71's, 188.997'MB/sec 2024/11/23-15:05:14.835309 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265295.sst): old prealloc_size = 147639500, real_size = 40453040 2024/11/23-15:05:14.835428 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265295.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.58 2024/11/23-15:05:14.835510 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265295.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:05:15.003445 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265295.sst): second pass time = 0.77's, 171.463'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 249.353'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2272.802'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.256'MB/sec write SST data time = 0.04's, 1010.253'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46585.942'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 171.463'MB/sec zip pipeline throughput = 171.463'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 310.58 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.41 avg-zval = 310.58 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 26301 bytes = 98640 } seq expand size = 184107 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.9938 value = 3.2745 dict = 1.00 all = 3.2018 } Zip/UnZip{ index = 0.1430 value = 0.3054 dict = 1.00 all = 0.3123 } ---------------------------- total value len = 11.363665 GB avg = 0.207 KB (by entry num) total key len = 1.490004 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009603 Billion total entry num = 0.054817332 Billion write speed all = 0.578921373 MB/sec (with seq num) write speed all = 0.559169861 MB/sec (without seq num) 2024/11/23-15:05:15.053158 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265295: 129431 keys, 41919230 bytes, temperature: kWarm 2024/11/23-15:05:15.053228 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345515053193, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265295, "file_size": 41919230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4050131901, "table_properties": {"data_size": 40198432, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131630117, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345513, "oldest_key_time": 0, "file_creation_time": 1732345513, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265295, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:15.101110 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265296.sst) /dev/shm/ToplingTemp/Topling-86185-LhvHW4 2024/11/23-15:05:15.637636 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265296.sst): first pass time = 0.54's, 250.180'MB/sec 2024/11/23-15:05:16.484763 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265296.sst): old prealloc_size = 147639500, real_size = 40612336 2024/11/23-15:05:16.484873 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265296.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.01 avg-zval = 311.82 2024/11/23-15:05:16.484967 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265296.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:05:16.649931 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265296.sst): second pass time = 0.85's, 155.726'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 255.384'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2402.709'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.265'MB/sec write SST data time = 0.04's, 978.716'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49120.197'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 155.726'MB/sec zip pipeline throughput = 157.027'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.01 avg-zval = 311.82 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1019.87 avg-zval = 311.82 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 52868 bytes = 198272 } seq expand size = 370076 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 6.9936 value = 3.2616 dict = 1.00 all = 3.1822 } Zip/UnZip{ index = 0.1430 value = 0.3066 dict = 1.00 all = 0.3143 } ---------------------------- total value len = 11.495664 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139031 Billion total entry num = 0.054946760 Billion write speed all = 0.584939764 MB/sec (with seq num) write speed all = 0.565143085 MB/sec (without seq num) 2024/11/23-15:05:16.711020 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265296: 129428 keys, 42178337 bytes, temperature: kWarm 2024/11/23-15:05:16.711086 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345516711051, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265296, "file_size": 42178337, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4050148285, "table_properties": {"data_size": 40357728, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629536, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345515, "oldest_key_time": 0, "file_creation_time": 1732345515, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265296, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:16.749185 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265297.sst) /dev/shm/ToplingTemp/Topling-86185-T18R07 2024/11/23-15:05:16.758104 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265297.sst): first pass time = 0.01's, 414.603'MB/sec 2024/11/23-15:05:16.792418 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265297.sst): old prealloc_size = 147639500, real_size = 1164712 2024/11/23-15:05:16.792457 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265297.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 3552 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.28 raw-val = 0.0036 GB zip-val = 0.0012 GB avg-val =1016.91 avg-zval = 327.33 2024/11/23-15:05:16.799083 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00/265297.sst): second pass time = 0.03's, 106.869'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.01's, 182.860'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 45.260'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.171'MB/sec write SST data time = 0.00's, 844.009'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.040'MB, throughput = 1515.572'MB/sec zip my value time = 0.03's, unzip length = 0.004'GB zip my value throughput = 106.869'MB/sec zip pipeline throughput = 106.869'MB/sec entries = 3552 avg-key = 20.00 avg-zkey = 0.29 avg-val = 1016.91 avg-zval = 327.33 usrkeys = 3552 avg-key = 12.00 avg-zkey = 0.29 avg-val = 1023.91 avg-zval = 327.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 3552 bytes = 11552 } seq expand size = 24864 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0000 GB value = 0.0036 GB dict = 0.04 MB all = 0.0037 GB } __ZipSize{ index = 0.0000 GB value = 0.0012 GB dict = 0.04 MB all = 0.0012 GB } UnZip/Zip{ index = 41.6250 value = 3.1067 dict = 1.00 all = 3.0286 } Zip/UnZip{ index = 0.0240 value = 0.3219 dict = 1.00 all = 0.3302 } ---------------------------- total value len = 11.499301 GB avg = 0.209 KB (by entry num) total key len = 1.492664 GB avg = 0.027 KB total ukey len = 1.028984 GB avg = 0.015 KB total ukey num = 0.068142583 Billion total entry num = 0.054950312 Billion write speed all = 0.585102826 MB/sec (with seq num) write speed all = 0.565305000 MB/sec (without seq num) 2024/11/23-15:05:16.810704 139957685155392 [compaction_job.cc:1948] [default] [JOB 28716] Generated table #265297: 3552 keys, 1216110 bytes, temperature: kWarm 2024/11/23-15:05:16.810749 139957685155392 EVENT_LOG_v1 {"time_micros": 1732345516810722, "cf_name": "default", "job": 28716, "event": "table_file_creation", "file_number": 265297, "file_size": 1216110, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4015350229, "largest_seqno": 4050147981, "table_properties": {"data_size": 1162672, "index_size": 1024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 71040, "raw_average_key_size": 20, "raw_value_size": 3612058, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 3552, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1732345516, "oldest_key_time": 0, "file_creation_time": 1732345516, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265297, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:05:16.819228 139957685155392 [compaction_job.cc:703] job-28716: subcompact[0], size: 0.850813 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28716/att-00 [265277,265278,265279,265280,265281,265282,265283,265284,265285,265286,265287,265288,265289,265290,265291,265292,265293,265294,265295,265296,265297] 2024/11/23-15:05:16.952801 139957685155392 (Original Log Time 2024/11/23-15:04:38.243502) EVENT_LOG_v1 {"time_micros": 1732345478243477, "job": 28716, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [265230], "files_L3": [265157, 265158, 265159, 265160, 265161, 265162, 265163, 265164, 265165, 265166, 265167, 265168, 265169, 265170, 265171, 265172, 265173, 265174, 265175, 265272], "score": 0.997776, "input_data_size": 882298869, "oldest_snapshot_seqno": 4051232730} 2024/11/23-15:05:16.952805 139957685155392 (Original Log Time 2024/11/23-15:05:16.951234) [compaction_job.cc:2022] [default] [JOB 28716] Compacted 1@2 + 20@3 files to L3 => 850813331 bytes 2024/11/23-15:05:16.952806 139957685155392 (Original Log Time 2024/11/23-15:05:16.952739) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 22.9 rd, 22.1 wr, level 3, files in(1, 20) out(21 +0 blob) MB in(31.2, 810.2 +0.0 blob) out(811.4 +0.0 blob), read-write-amplify(53.0) write-amplify(26.0) OK, records in: 2620503, records dropped: 28355 output_compression: Snappy 2024/11/23-15:05:16.952809 139957685155392 (Original Log Time 2024/11/23-15:05:16.952780) EVENT_LOG_v1 {"time_micros": 1732345516952761, "cf": "default", "job": 28716, "event": "compaction_finished", "compaction_time_micros": 38575733, "compaction_time_cpu_micros": 7537755, "output_level": 3, "num_output_files": 21, "total_output_size": 850813331, "num_input_records": 2620503, "num_output_records": 2592148, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 669849243, "file_range_sync_nanos": 0, "file_fsync_nanos": 14486, "file_prepare_write_nanos": 47236, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/23-15:05:17.080843 139957685155392 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732345478,"executesMs":38854,"compactionJobId":28716,"attempt":0,"compactionInputRawBytes":2717452336,"compactionInputZipBytes":882298869,"compactionOutputRawBytes":0,"compactionOutputZipBytes":850813331,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-d3a157d0a56b9c1e49c4f44471215ac1-a6212c7f8610a345-00"}}, response = 2024/11/23-15:05:17.080919 139957685155392 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28716/att-00: olev 3, work 38.855 s, result 7.801 ms, install 1.590 ms, input{raw 2.531 GiB zip 841.426 MiB}