2024/11/24-03:52:16.918344 139957215163968 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00: fork to child time = 0.021883 sec 2024/11/24-03:52:16.919597 139957215163968 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00 2024/11/24-03:52:16.929014 139957215163968 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00 2024/11/24-03:52:16.930377 139957215163968 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-03:52:16.930746 139957215163968 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-03:52:16.985717 139957215163968 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41096/att-00/MANIFEST-206347 2024/11/24-03:52:17.021279 139957215163968 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41096/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 290988, last_sequence is 4426433166, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 290816 2024/11/24-03:52:17.021291 139957215163968 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-03:52:17.021488 139957215163968 [version_set.cc:5721] Creating manifest 290988 2024/11/24-03:52:17.056965 139957215163968 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-03:52:17.077529 139957215163968 [compaction_job.cc:2437] [default] [JOB 41096] Compacting 1@2 + 17@3 files to L3, score 0.91, subcompactions 1 : 1 2024/11/24-03:52:17.077559 139957215163968 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 30778 Base level 2, inputs: [290963(27MB)], [290872(35MB) 290873(35MB) 290874(33MB) 290875(34MB) 290876(35MB) 290877(34MB) 290878(33MB) 290879(35MB) 290880(35MB) 290881(34MB) 290882(33MB) 290883(35MB) 290884(22MB) 290885(33MB) 290886(35MB) 290915(35MB) 290972(33MB)] 2024/11/24-03:52:17.084374 139957215163968 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:52:17.084404 139957215163968 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:52:17.084424 139957215163968 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:52:17.084430 139957215163968 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:52:17.084442 139957215163968 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:52:17.661002 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290989.sst) /dev/shm/ToplingTemp/Topling-262860-2VLiCs 2024/11/24-03:52:18.212583 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290989.sst): first pass time = 0.55's, 243.349'MB/sec 2024/11/24-03:52:20.016754 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290989.sst): old prealloc_size = 147639500, real_size = 33380336 2024/11/24-03:52:20.017463 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290989.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926911 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.80 avg-zval = 33.86 2024/11/24-03:52:20.018346 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290989.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:21.111853 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290989.sst): second pass time = 1.80's, 60.038'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 34.224'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2580.367'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.246'MB/sec write SST data time = 0.15's, 248.110'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31476.614'MB/sec zip my value time = 1.80's, unzip length = 0.108'GB zip my value throughput = 60.038'MB/sec zip pipeline throughput = 89.471'MB/sec entries = 926911 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 33.86 usrkeys = 926911 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 33.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926911 bytes = 3244208 } seq expand size = 6488377 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5128 value = 3.4495 dict = 1.00 all = 3.5812 } Zip/UnZip{ index = 0.0951 value = 0.2899 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 2251.440326 GB avg = 0.200 KB (by entry num) total key len = 306.908916 GB avg = 0.027 KB total ukey len = 237.373266 GB avg = 0.015 KB total ukey num = 15.739588074 Billion total entry num = 11.280902022 Billion write speed all = 37.496564457 MB/sec (with seq num) write speed all = 36.173851943 MB/sec (without seq num) 2024/11/24-03:52:21.157238 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #290989: 926911 keys, 37478792 bytes, temperature: kWarm 2024/11/24-03:52:21.157319 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391541157265, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 290989, "file_size": 37478792, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4259662645, "largest_seqno": 4426239746, "table_properties": {"data_size": 31385056, "index_size": 1763392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953508, "raw_average_key_size": 28, "raw_value_size": 108264238, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926911, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391537, "oldest_key_time": 0, "file_creation_time": 1732391537, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290989, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:21.194797 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290990.sst) /dev/shm/ToplingTemp/Topling-262860-TyLrqq 2024/11/24-03:52:22.130479 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290990.sst): first pass time = 0.94's, 143.449'MB/sec 2024/11/24-03:52:23.643133 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290990.sst): old prealloc_size = 147639500, real_size = 32790576 2024/11/24-03:52:23.643679 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290990.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927037 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 116.78 avg-zval = 33.22 2024/11/24-03:52:23.644468 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290990.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:24.497433 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290990.sst): second pass time = 1.51's, 71.635'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.183'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3352.030'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 285.429'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28032.638'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.635'MB/sec zip pipeline throughput = 85.220'MB/sec entries = 927037 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 33.22 usrkeys = 927037 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.78 avg-zval = 33.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927037 bytes = 3244640 } seq expand size = 6489259 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5135 value = 3.5155 dict = 1.00 all = 3.6383 } Zip/UnZip{ index = 0.0951 value = 0.2845 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 2251.555076 GB avg = 0.200 KB (by entry num) total key len = 306.934873 GB avg = 0.027 KB total ukey len = 237.391806 GB avg = 0.015 KB total ukey num = 15.740515111 Billion total entry num = 11.281829059 Billion write speed all = 37.496766117 MB/sec (with seq num) write speed all = 36.174010542 MB/sec (without seq num) 2024/11/24-03:52:24.544722 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #290990: 927037 keys, 36889786 bytes, temperature: kWarm 2024/11/24-03:52:24.544794 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391544544759, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 290990, "file_size": 36889786, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178503, "largest_seqno": 4426239052, "table_properties": {"data_size": 30795104, "index_size": 1763520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957036, "raw_average_key_size": 28, "raw_value_size": 108260794, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927037, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391541, "oldest_key_time": 0, "file_creation_time": 1732391541, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290990, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:24.581050 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290991.sst) /dev/shm/ToplingTemp/Topling-262860-wl2RGr 2024/11/24-03:52:25.470513 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290991.sst): first pass time = 0.89's, 150.904'MB/sec 2024/11/24-03:52:26.884777 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290991.sst): old prealloc_size = 147639500, real_size = 31132384 2024/11/24-03:52:26.885338 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290991.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923163 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.39 avg-zval = 31.56 2024/11/24-03:52:26.886075 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290991.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:27.695016 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290991.sst): second pass time = 1.41's, 76.666'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.328'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3287.302'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.338'MB/sec write SST data time = 0.13's, 277.438'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35769.828'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.666'MB/sec zip pipeline throughput = 76.666'MB/sec entries = 923163 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.39 avg-zval = 31.56 usrkeys = 923163 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.39 avg-zval = 31.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923163 bytes = 3115696 } seq expand size = 6462141 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4430 value = 3.7198 dict = 1.00 all = 3.8234 } Zip/UnZip{ index = 0.0958 value = 0.2688 dict = 1.00 all = 0.2615 } ---------------------------- total value len = 2251.669907 GB avg = 0.200 KB (by entry num) total key len = 306.960722 GB avg = 0.027 KB total ukey len = 237.410270 GB avg = 0.015 KB total ukey num = 15.741438274 Billion total entry num = 11.282752222 Billion write speed all = 37.497070671 MB/sec (with seq num) write speed all = 36.174268848 MB/sec (without seq num) 2024/11/24-03:52:27.738432 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #290991: 923163 keys, 35104060 bytes, temperature: kWarm 2024/11/24-03:52:27.738500 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391547738467, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 290991, "file_size": 35104060, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4296556816, "largest_seqno": 4426238064, "table_properties": {"data_size": 29133392, "index_size": 1768000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25848564, "raw_average_key_size": 28, "raw_value_size": 108369250, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923163, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391544, "oldest_key_time": 0, "file_creation_time": 1732391544, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290991, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:27.774190 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290992.sst) /dev/shm/ToplingTemp/Topling-262860-pmovYp 2024/11/24-03:52:28.748585 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290992.sst): first pass time = 0.97's, 137.752'MB/sec 2024/11/24-03:52:30.187370 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290992.sst): old prealloc_size = 147639500, real_size = 32711968 2024/11/24-03:52:30.187896 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290992.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923013 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0307 GB avg-val = 117.41 avg-zval = 33.29 2024/11/24-03:52:30.188660 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290992.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:31.014767 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290992.sst): second pass time = 1.44's, 75.397'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.478'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3484.388'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.12's, 298.100'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33985.748'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.397'MB/sec zip pipeline throughput = 75.397'MB/sec entries = 923013 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.41 avg-zval = 33.29 usrkeys = 923013 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.41 avg-zval = 33.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923013 bytes = 3230560 } seq expand size = 6461091 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5140 value = 3.5272 dict = 1.00 all = 3.6472 } Zip/UnZip{ index = 0.0951 value = 0.2835 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 2251.784742 GB avg = 0.200 KB (by entry num) total key len = 306.986566 GB avg = 0.027 KB total ukey len = 237.428730 GB avg = 0.015 KB total ukey num = 15.742361287 Billion total entry num = 11.283675235 Billion write speed all = 37.497308033 MB/sec (with seq num) write speed all = 36.174462354 MB/sec (without seq num) 2024/11/24-03:52:31.060567 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #290992: 923013 keys, 36800234 bytes, temperature: kWarm 2024/11/24-03:52:31.060636 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391551060601, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 290992, "file_size": 36800234, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178518, "largest_seqno": 4426239452, "table_properties": {"data_size": 30725264, "index_size": 1755776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25844364, "raw_average_key_size": 28, "raw_value_size": 108373418, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923013, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391547, "oldest_key_time": 0, "file_creation_time": 1732391547, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290992, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:31.099267 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290993.sst) /dev/shm/ToplingTemp/Topling-262860-ANwIvq 2024/11/24-03:52:32.126415 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290993.sst): first pass time = 1.03's, 130.674'MB/sec 2024/11/24-03:52:33.609087 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290993.sst): old prealloc_size = 147639500, real_size = 33396944 2024/11/24-03:52:33.609663 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290993.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926971 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.79 avg-zval = 33.86 2024/11/24-03:52:33.610436 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290993.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:34.483795 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290993.sst): second pass time = 1.48's, 73.056'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.868'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3203.407'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.13's, 296.306'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35003.952'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.056'MB/sec zip pipeline throughput = 73.056'MB/sec entries = 926971 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.79 avg-zval = 33.86 usrkeys = 926971 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.79 avg-zval = 33.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926971 bytes = 3244416 } seq expand size = 6488797 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4445 value = 3.4490 dict = 1.00 all = 3.5795 } Zip/UnZip{ index = 0.0957 value = 0.2899 dict = 1.00 all = 0.2794 } ---------------------------- total value len = 2251.899493 GB avg = 0.200 KB (by entry num) total key len = 307.012521 GB avg = 0.027 KB total ukey len = 237.447269 GB avg = 0.015 KB total ukey num = 15.743288258 Billion total entry num = 11.284602206 Billion write speed all = 37.497463765 MB/sec (with seq num) write speed all = 36.174576663 MB/sec (without seq num) 2024/11/24-03:52:34.528920 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #290993: 926971 keys, 37495942 bytes, temperature: kWarm 2024/11/24-03:52:34.528996 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391554528954, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 290993, "file_size": 37495942, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178530, "largest_seqno": 4426238840, "table_properties": {"data_size": 31389952, "index_size": 1775040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955188, "raw_average_key_size": 28, "raw_value_size": 108262546, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926971, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391551, "oldest_key_time": 0, "file_creation_time": 1732391551, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290993, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:34.563768 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290994.sst) /dev/shm/ToplingTemp/Topling-262860-sANU9q 2024/11/24-03:52:35.494533 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290994.sst): first pass time = 0.93's, 144.208'MB/sec 2024/11/24-03:52:36.952136 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290994.sst): old prealloc_size = 147639500, real_size = 31978400 2024/11/24-03:52:36.952714 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290994.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926894 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0300 GB avg-val = 116.80 avg-zval = 32.35 2024/11/24-03:52:36.953498 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290994.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:37.785408 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290994.sst): second pass time = 1.46's, 74.320'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.297'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3180.402'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 282.648'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30891.621'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.320'MB/sec zip pipeline throughput = 74.320'MB/sec entries = 926894 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 32.35 usrkeys = 926894 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 32.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926894 bytes = 3244144 } seq expand size = 6488258 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5119 value = 3.6109 dict = 1.00 all = 3.7203 } Zip/UnZip{ index = 0.0951 value = 0.2769 dict = 1.00 all = 0.2688 } ---------------------------- total value len = 2252.014246 GB avg = 0.200 KB (by entry num) total key len = 307.038474 GB avg = 0.027 KB total ukey len = 237.465807 GB avg = 0.015 KB total ukey num = 15.744215152 Billion total entry num = 11.285529100 Billion write speed all = 37.497711457 MB/sec (with seq num) write speed all = 36.174779701 MB/sec (without seq num) 2024/11/24-03:52:37.829657 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #290994: 926894 keys, 36076904 bytes, temperature: kWarm 2024/11/24-03:52:37.829729 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391557829691, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 290994, "file_size": 36076904, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178548, "largest_seqno": 4426239801, "table_properties": {"data_size": 29982992, "index_size": 1763520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953032, "raw_average_key_size": 28, "raw_value_size": 108264716, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926894, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391554, "oldest_key_time": 0, "file_creation_time": 1732391554, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290994, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:37.864417 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290995.sst) /dev/shm/ToplingTemp/Topling-262860-hUkU2q 2024/11/24-03:52:38.776581 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290995.sst): first pass time = 0.91's, 147.147'MB/sec 2024/11/24-03:52:40.199569 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290995.sst): old prealloc_size = 147639500, real_size = 31400640 2024/11/24-03:52:40.200131 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290995.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919948 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 117.90 avg-zval = 31.97 2024/11/24-03:52:40.200889 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290995.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:41.011039 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290995.sst): second pass time = 1.42's, 76.261'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.725'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3268.714'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 282.893'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35663.872'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.261'MB/sec zip pipeline throughput = 76.261'MB/sec entries = 919948 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.90 avg-zval = 31.97 usrkeys = 919948 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.90 avg-zval = 31.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919948 bytes = 3219840 } seq expand size = 6439636 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4467 value = 3.6879 dict = 1.00 all = 3.7830 } Zip/UnZip{ index = 0.0957 value = 0.2712 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2252.129145 GB avg = 0.200 KB (by entry num) total key len = 307.064233 GB avg = 0.027 KB total ukey len = 237.484206 GB avg = 0.015 KB total ukey num = 15.745135100 Billion total entry num = 11.286449048 Billion write speed all = 37.498000173 MB/sec (with seq num) write speed all = 36.175023107 MB/sec (without seq num) 2024/11/24-03:52:41.054396 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #290995: 919948 keys, 35479086 bytes, temperature: kWarm 2024/11/24-03:52:41.054464 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391561054429, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 290995, "file_size": 35479086, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4277898032, "largest_seqno": 4426238110, "table_properties": {"data_size": 29409264, "index_size": 1761216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25758544, "raw_average_key_size": 28, "raw_value_size": 108459312, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919948, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391557, "oldest_key_time": 0, "file_creation_time": 1732391557, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290995, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:41.089260 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290996.sst) /dev/shm/ToplingTemp/Topling-262860-yio8ks 2024/11/24-03:52:42.115917 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290996.sst): first pass time = 1.03's, 130.737'MB/sec 2024/11/24-03:52:43.586065 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290996.sst): old prealloc_size = 147639500, real_size = 33053920 2024/11/24-03:52:43.586607 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290996.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926197 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.91 avg-zval = 33.54 2024/11/24-03:52:43.587390 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290996.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:44.437540 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290996.sst): second pass time = 1.47's, 73.714'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.631'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3402.308'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 296.494'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30987.836'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.714'MB/sec zip pipeline throughput = 73.714'MB/sec entries = 926197 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 33.54 usrkeys = 926197 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.91 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926197 bytes = 3241712 } seq expand size = 6483379 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5131 value = 3.4863 dict = 1.00 all = 3.6129 } Zip/UnZip{ index = 0.0951 value = 0.2868 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 2252.243913 GB avg = 0.200 KB (by entry num) total key len = 307.090166 GB avg = 0.027 KB total ukey len = 237.502730 GB avg = 0.015 KB total ukey num = 15.746061297 Billion total entry num = 11.287375245 Billion write speed all = 37.498179117 MB/sec (with seq num) write speed all = 36.175159908 MB/sec (without seq num) 2024/11/24-03:52:44.483686 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #290996: 926197 keys, 37150014 bytes, temperature: kWarm 2024/11/24-03:52:44.483757 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391564483721, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 290996, "file_size": 37150014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178559, "largest_seqno": 4426236312, "table_properties": {"data_size": 31060176, "index_size": 1761984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933516, "raw_average_key_size": 28, "raw_value_size": 108284286, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926197, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391561, "oldest_key_time": 0, "file_creation_time": 1732391561, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290996, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:44.520323 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290997.sst) /dev/shm/ToplingTemp/Topling-262860-TOejyr 2024/11/24-03:52:45.511716 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290997.sst): first pass time = 0.99's, 135.388'MB/sec 2024/11/24-03:52:47.105427 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290997.sst): old prealloc_size = 147639500, real_size = 33282976 2024/11/24-03:52:47.105958 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290997.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926955 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.79 avg-zval = 33.75 2024/11/24-03:52:47.106816 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290997.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:47.968439 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290997.sst): second pass time = 1.59's, 67.965'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.317'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3482.237'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 296.215'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32975.462'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 67.965'MB/sec zip pipeline throughput = 112.677'MB/sec entries = 926955 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 33.75 usrkeys = 926955 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.79 avg-zval = 33.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926955 bytes = 3244352 } seq expand size = 6488685 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5137 value = 3.4602 dict = 1.00 all = 3.5905 } Zip/UnZip{ index = 0.0951 value = 0.2890 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 2252.358664 GB avg = 0.200 KB (by entry num) total key len = 307.116121 GB avg = 0.027 KB total ukey len = 237.521269 GB avg = 0.015 KB total ukey num = 15.746988252 Billion total entry num = 11.288302200 Billion write speed all = 37.498300781 MB/sec (with seq num) write speed all = 36.175241367 MB/sec (without seq num) 2024/11/24-03:52:48.034758 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #290997: 926955 keys, 37381866 bytes, temperature: kWarm 2024/11/24-03:52:48.034846 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391568034798, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 290997, "file_size": 37381866, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178575, "largest_seqno": 4426238876, "table_properties": {"data_size": 31287696, "index_size": 1763328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954740, "raw_average_key_size": 28, "raw_value_size": 108263066, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926955, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391564, "oldest_key_time": 0, "file_creation_time": 1732391564, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290997, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:48.080928 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290998.sst) /dev/shm/ToplingTemp/Topling-262860-nWwQqq 2024/11/24-03:52:48.975465 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290998.sst): first pass time = 0.89's, 150.049'MB/sec 2024/11/24-03:52:50.493618 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290998.sst): old prealloc_size = 147639500, real_size = 30771936 2024/11/24-03:52:50.494159 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290998.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926506 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.86 avg-zval = 31.05 2024/11/24-03:52:50.494913 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290998.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:51.307804 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290998.sst): second pass time = 1.52's, 71.352'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.684'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3422.956'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.338'MB/sec write SST data time = 0.13's, 268.667'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34142.361'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.352'MB/sec zip pipeline throughput = 125.215'MB/sec entries = 926506 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.86 avg-zval = 31.05 usrkeys = 926506 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.86 avg-zval = 31.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926506 bytes = 3126976 } seq expand size = 6485542 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4438 value = 3.7640 dict = 1.00 all = 3.8622 } Zip/UnZip{ index = 0.0958 value = 0.2657 dict = 1.00 all = 0.2589 } ---------------------------- total value len = 2252.473426 GB avg = 0.200 KB (by entry num) total key len = 307.142063 GB avg = 0.027 KB total ukey len = 237.539799 GB avg = 0.015 KB total ukey num = 15.747914758 Billion total entry num = 11.289228706 Billion write speed all = 37.498527602 MB/sec (with seq num) write speed all = 36.175424328 MB/sec (without seq num) 2024/11/24-03:52:51.384787 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #290998: 926506 keys, 34751922 bytes, temperature: kWarm 2024/11/24-03:52:51.384847 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391571384815, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 290998, "file_size": 34751922, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4316565274, "largest_seqno": 4426238142, "table_properties": {"data_size": 28765840, "index_size": 1774272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942168, "raw_average_key_size": 28, "raw_value_size": 108275588, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926506, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391568, "oldest_key_time": 0, "file_creation_time": 1732391568, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290998, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:51.423476 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290999.sst) /dev/shm/ToplingTemp/Topling-262860-1cQhIq 2024/11/24-03:52:52.279234 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290999.sst): first pass time = 0.86's, 156.848'MB/sec 2024/11/24-03:52:53.890291 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290999.sst): old prealloc_size = 147639500, real_size = 31483088 2024/11/24-03:52:53.890818 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290999.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919607 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.95 avg-zval = 32.08 2024/11/24-03:52:53.891616 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290999.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:54.705001 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/290999.sst): second pass time = 1.61's, 67.366'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.653'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3495.939'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 278.281'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 29073.716'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.366'MB/sec zip pipeline throughput = 128.762'MB/sec entries = 919607 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.95 avg-zval = 32.08 usrkeys = 919607 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.95 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919607 bytes = 3218640 } seq expand size = 6437249 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5139 value = 3.6764 dict = 1.00 all = 3.7741 } Zip/UnZip{ index = 0.0951 value = 0.2720 dict = 1.00 all = 0.2650 } ---------------------------- total value len = 2252.588332 GB avg = 0.200 KB (by entry num) total key len = 307.167812 GB avg = 0.027 KB total ukey len = 237.558191 GB avg = 0.015 KB total ukey num = 15.748834365 Billion total entry num = 11.290148313 Billion write speed all = 37.498721941 MB/sec (with seq num) write speed all = 36.175576739 MB/sec (without seq num) 2024/11/24-03:52:54.756390 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #290999: 919607 keys, 35562586 bytes, temperature: kWarm 2024/11/24-03:52:54.756448 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391574756415, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 290999, "file_size": 35562586, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178589, "largest_seqno": 4426234601, "table_properties": {"data_size": 29503680, "index_size": 1749312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25748996, "raw_average_key_size": 28, "raw_value_size": 108468762, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919607, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391571, "oldest_key_time": 0, "file_creation_time": 1732391571, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290999, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:54.796236 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291000.sst) /dev/shm/ToplingTemp/Topling-262860-K7HuEp 2024/11/24-03:52:55.684890 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291000.sst): first pass time = 0.89's, 151.040'MB/sec 2024/11/24-03:52:57.261657 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291000.sst): old prealloc_size = 147639500, real_size = 33394640 2024/11/24-03:52:57.262197 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291000.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926891 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.80 avg-zval = 33.88 2024/11/24-03:52:57.262960 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291000.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:52:58.119392 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291000.sst): second pass time = 1.58's, 68.695'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.713'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3340.643'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 297.476'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27107.464'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.695'MB/sec zip pipeline throughput = 117.385'MB/sec entries = 926891 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 33.88 usrkeys = 926891 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 33.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926891 bytes = 3244128 } seq expand size = 6488237 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5126 value = 3.4480 dict = 1.00 all = 3.5798 } Zip/UnZip{ index = 0.0951 value = 0.2900 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 2252.703085 GB avg = 0.200 KB (by entry num) total key len = 307.193765 GB avg = 0.027 KB total ukey len = 237.576729 GB avg = 0.015 KB total ukey num = 15.749761256 Billion total entry num = 11.291075204 Billion write speed all = 37.498907537 MB/sec (with seq num) write speed all = 36.175719894 MB/sec (without seq num) 2024/11/24-03:52:58.165054 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #291000: 926891 keys, 37493004 bytes, temperature: kWarm 2024/11/24-03:52:58.165121 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391578165089, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 291000, "file_size": 37493004, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4426239271, "table_properties": {"data_size": 31399360, "index_size": 1763392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952948, "raw_average_key_size": 28, "raw_value_size": 108264814, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926891, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391574, "oldest_key_time": 0, "file_creation_time": 1732391574, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291000, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:52:58.201572 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291001.sst) /dev/shm/ToplingTemp/Topling-262860-orj2Hq 2024/11/24-03:52:59.296610 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291001.sst): first pass time = 1.09's, 93.408'MB/sec 2024/11/24-03:53:00.447991 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291001.sst): old prealloc_size = 147639500, real_size = 24260768 2024/11/24-03:53:00.448450 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291001.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 708243 prefix = 4 raw-key = 0.0113 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0825 GB zip-val = 0.0227 GB avg-val = 116.42 avg-zval = 32.09 2024/11/24-03:53:00.449066 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291001.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0018 GB, waited 0.000 sec, Key+Value = 0.102 GB 2024/11/24-03:53:01.106735 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291001.sst): second pass time = 1.15's, 71.658'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.66's, 41.452'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3115.230'MB/sec (index lex order gen) rebuild zvType time = 0.56's, 0.316'MB/sec write SST data time = 0.10's, 279.920'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.828'MB, throughput = 28628.202'MB/sec zip my value time = 1.15's, unzip length = 0.082'GB zip my value throughput = 71.658'MB/sec zip pipeline throughput = 71.658'MB/sec entries = 708243 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.42 avg-zval = 32.09 usrkeys = 708243 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.42 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 708243 bytes = 2390336 } seq expand size = 4957701 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0142 GB value = 0.0825 GB dict = 0.83 MB all = 0.1023 GB } __ZipSize{ index = 0.0014 GB value = 0.0227 GB dict = 0.83 MB all = 0.0273 GB } UnZip/Zip{ index = 10.4424 value = 3.6279 dict = 1.00 all = 3.7458 } Zip/UnZip{ index = 0.0958 value = 0.2756 dict = 1.00 all = 0.2670 } ---------------------------- total value len = 2252.790493 GB avg = 0.200 KB (by entry num) total key len = 307.213596 GB avg = 0.027 KB total ukey len = 237.590894 GB avg = 0.015 KB total ukey num = 15.750469499 Billion total entry num = 11.291783447 Billion write speed all = 37.498837473 MB/sec (with seq num) write speed all = 36.175624735 MB/sec (without seq num) 2024/11/24-03:53:01.143433 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #291001: 708243 keys, 27305520 bytes, temperature: kWarm 2024/11/24-03:53:01.143501 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391581143467, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 291001, "file_size": 27305520, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4355609689, "largest_seqno": 4426239358, "table_properties": {"data_size": 22727056, "index_size": 1356480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 19830804, "raw_average_key_size": 28, "raw_value_size": 82450398, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 708243, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391578, "oldest_key_time": 0, "file_creation_time": 1732391578, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291001, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:01.176420 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291002.sst) /dev/shm/ToplingTemp/Topling-262860-VMWKUs 2024/11/24-03:53:02.175923 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291002.sst): first pass time = 1.00's, 134.291'MB/sec 2024/11/24-03:53:03.751785 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291002.sst): old prealloc_size = 147639500, real_size = 31398672 2024/11/24-03:53:03.752366 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291002.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.99 2024/11/24-03:53:03.753140 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291002.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:53:04.570804 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291002.sst): second pass time = 1.57's, 68.878'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.316'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3154.402'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 280.378'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33874.805'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.878'MB/sec zip pipeline throughput = 98.409'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.99 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6886 dict = 1.00 all = 3.7834 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2252.905405 GB avg = 0.200 KB (by entry num) total key len = 307.239336 GB avg = 0.027 KB total ukey len = 237.609280 GB avg = 0.015 KB total ukey num = 15.751388799 Billion total entry num = 11.292702747 Billion write speed all = 37.498994997 MB/sec (with seq num) write speed all = 36.175741676 MB/sec (without seq num) 2024/11/24-03:53:04.614472 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #291002: 919300 keys, 35475378 bytes, temperature: kWarm 2024/11/24-03:53:04.614543 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391584614508, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 291002, "file_size": 35475378, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4414357721, "table_properties": {"data_size": 29408512, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391581, "oldest_key_time": 0, "file_creation_time": 1732391581, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291002, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:04.657543 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291003.sst) /dev/shm/ToplingTemp/Topling-262860-NRG80q 2024/11/24-03:53:05.241880 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291003.sst): first pass time = 0.58's, 229.714'MB/sec 2024/11/24-03:53:06.653397 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291003.sst): old prealloc_size = 147639500, real_size = 33249872 2024/11/24-03:53:06.654093 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291003.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926857 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.81 avg-zval = 33.72 2024/11/24-03:53:06.654932 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291003.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:53:07.567446 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291003.sst): second pass time = 1.41's, 76.755'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.861'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2615.049'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.15's, 241.591'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35772.398'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.755'MB/sec zip pipeline throughput = 76.755'MB/sec entries = 926857 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 33.72 usrkeys = 926857 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.81 avg-zval = 33.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926857 bytes = 3244016 } seq expand size = 6487999 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5111 value = 3.4640 dict = 1.00 all = 3.5937 } Zip/UnZip{ index = 0.0951 value = 0.2887 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 2253.020159 GB avg = 0.199 KB (by entry num) total key len = 307.265288 GB avg = 0.027 KB total ukey len = 237.627817 GB avg = 0.015 KB total ukey num = 15.752315656 Billion total entry num = 11.293629604 Billion write speed all = 37.499410002 MB/sec (with seq num) write speed all = 36.176106158 MB/sec (without seq num) 2024/11/24-03:53:07.612860 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #291003: 926857 keys, 37348060 bytes, temperature: kWarm 2024/11/24-03:53:07.612933 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391587612896, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 291003, "file_size": 37348060, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178655, "largest_seqno": 4426239684, "table_properties": {"data_size": 31254400, "index_size": 1763584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951996, "raw_average_key_size": 28, "raw_value_size": 108265774, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926857, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391584, "oldest_key_time": 0, "file_creation_time": 1732391584, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291003, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:07.663106 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291004.sst) /dev/shm/ToplingTemp/Topling-262860-RLihzs 2024/11/24-03:53:08.642567 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291004.sst): first pass time = 0.98's, 137.038'MB/sec 2024/11/24-03:53:10.137043 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291004.sst): old prealloc_size = 147639500, real_size = 32936576 2024/11/24-03:53:10.137584 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291004.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926599 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.85 avg-zval = 33.38 2024/11/24-03:53:10.138344 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291004.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:53:10.984803 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291004.sst): second pass time = 1.49's, 72.489'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.687'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3398.828'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 293.473'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32932.126'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.489'MB/sec zip pipeline throughput = 72.489'MB/sec entries = 926599 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.85 avg-zval = 33.38 usrkeys = 926599 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.85 avg-zval = 33.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926599 bytes = 3243104 } seq expand size = 6486193 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4433 value = 3.5006 dict = 1.00 all = 3.6241 } Zip/UnZip{ index = 0.0958 value = 0.2857 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 2253.134918 GB avg = 0.199 KB (by entry num) total key len = 307.291233 GB avg = 0.027 KB total ukey len = 237.646349 GB avg = 0.015 KB total ukey num = 15.753242255 Billion total entry num = 11.294556203 Billion write speed all = 37.499593883 MB/sec (with seq num) write speed all = 36.176247703 MB/sec (without seq num) 2024/11/24-03:53:11.030261 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #291004: 926599 keys, 37034408 bytes, temperature: kWarm 2024/11/24-03:53:11.030332 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391591030296, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 291004, "file_size": 37034408, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4426239488, "table_properties": {"data_size": 30930224, "index_size": 1774528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25944772, "raw_average_key_size": 28, "raw_value_size": 108273042, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926599, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391587, "oldest_key_time": 0, "file_creation_time": 1732391587, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291004, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:11.067956 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291005.sst) /dev/shm/ToplingTemp/Topling-262860-tcYoBs 2024/11/24-03:53:11.942001 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291005.sst): first pass time = 0.87's, 153.569'MB/sec 2024/11/24-03:53:13.370991 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291005.sst): old prealloc_size = 147639500, real_size = 31112896 2024/11/24-03:53:13.371544 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291005.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924233 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0291 GB avg-val = 117.22 avg-zval = 31.50 2024/11/24-03:53:13.372337 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291005.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:53:14.199396 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291005.sst): second pass time = 1.43's, 75.855'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.356'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3330.262'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 270.501'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34028.558'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.855'MB/sec zip pipeline throughput = 75.855'MB/sec entries = 924233 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.22 avg-zval = 31.50 usrkeys = 924233 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.22 avg-zval = 31.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924233 bytes = 3119296 } seq expand size = 6469631 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4449 value = 3.7215 dict = 1.00 all = 3.8253 } Zip/UnZip{ index = 0.0957 value = 0.2687 dict = 1.00 all = 0.2614 } ---------------------------- total value len = 2253.249727 GB avg = 0.199 KB (by entry num) total key len = 307.317112 GB avg = 0.027 KB total ukey len = 237.664834 GB avg = 0.015 KB total ukey num = 15.754166488 Billion total entry num = 11.295480436 Billion write speed all = 37.499888856 MB/sec (with seq num) write speed all = 36.176496693 MB/sec (without seq num) 2024/11/24-03:53:14.243076 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #291005: 924233 keys, 35087074 bytes, temperature: kWarm 2024/11/24-03:53:14.243142 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391594243106, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 291005, "file_size": 35087074, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4303074310, "largest_seqno": 4426238296, "table_properties": {"data_size": 29111920, "index_size": 1769728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25878524, "raw_average_key_size": 28, "raw_value_size": 108339262, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924233, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391591, "oldest_key_time": 0, "file_creation_time": 1732391591, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291005, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:14.284994 139957215163968 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291006.sst) /dev/shm/ToplingTemp/Topling-262860-s7gQ5o 2024/11/24-03:53:14.301099 139957215163968 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291006.sst): first pass time = 0.02's, 181.655'MB/sec 2024/11/24-03:53:14.342751 139957215163968 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291006.sst): old prealloc_size = 147639500, real_size = 919312 2024/11/24-03:53:14.342814 139957215163968 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291006.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 20543 prefix = 4 raw-key = 0.0003 GB zip-key = 0.0000 GB avg-key = 16.00 avg-zkey = 2.00 raw-val = 0.0023 GB zip-val = 0.0009 GB avg-val = 114.00 avg-zval = 42.49 2024/11/24-03:53:14.342865 139957215163968 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291006.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0000 GB, waited 0.000 sec, Key+Value = 0.003 GB 2024/11/24-03:53:14.359347 139957215163968 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00/291006.sst): second pass time = 0.04's, 57.010'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.02's, 60.012'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 955.845'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.386'MB/sec write SST data time = 0.00's, 308.184'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.023'MB, throughput = 1088.280'MB/sec zip my value time = 0.04's, unzip length = 0.002'GB zip my value throughput = 57.010'MB/sec zip pipeline throughput = 57.010'MB/sec entries = 20543 avg-key = 28.00 avg-zkey = 2.00 avg-val = 114.00 avg-zval = 42.49 usrkeys = 20543 avg-key = 20.00 avg-zkey = 2.00 avg-val = 121.00 avg-zval = 42.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 20543 bytes = 56512 } seq expand size = 143801 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0004 GB value = 0.0023 GB dict = 0.02 MB all = 0.0029 GB } __ZipSize{ index = 0.0000 GB value = 0.0009 GB dict = 0.02 MB all = 0.0010 GB } UnZip/Zip{ index = 9.9840 value = 2.6831 dict = 1.00 all = 2.9305 } Zip/UnZip{ index = 0.1002 value = 0.3727 dict = 1.00 all = 0.3412 } ---------------------------- total value len = 2253.252213 GB avg = 0.199 KB (by entry num) total key len = 307.317687 GB avg = 0.027 KB total ukey len = 237.665245 GB avg = 0.015 KB total ukey num = 15.754187031 Billion total entry num = 11.295500979 Billion write speed all = 37.499845830 MB/sec (with seq num) write speed all = 36.176454361 MB/sec (without seq num) 2024/11/24-03:53:14.368047 139957215163968 [compaction_job.cc:1948] [default] [JOB 41096] Generated table #291006: 20543 keys, 995413 bytes, temperature: kWarm 2024/11/24-03:53:14.368096 139957215163968 EVENT_LOG_v1 {"time_micros": 1732391594368068, "cf_name": "default", "job": 41096, "event": "table_file_creation", "file_number": 291006, "file_size": 995413, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4412701634, "largest_seqno": 4415797479, "table_properties": {"data_size": 872832, "index_size": 41152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 575204, "raw_average_key_size": 28, "raw_value_size": 2341902, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 20543, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732391594, "oldest_key_time": 0, "file_creation_time": 1732391594, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 291006, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:53:14.378501 139957215163968 [compaction_job.cc:703] job-41096: subcompact[0], size: 0.610910 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41096/att-00 [290989,290990,290991,290992,290993,290994,290995,290996,290997,290998,290999,291000,291001,291002,291003,291004,291005,291006] 2024/11/24-03:53:14.485238 139957215163968 (Original Log Time 2024/11/24-03:52:17.077643) EVENT_LOG_v1 {"time_micros": 1732391537077582, "job": 41096, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [290963], "files_L3": [290872, 290873, 290874, 290875, 290876, 290877, 290878, 290879, 290880, 290881, 290882, 290883, 290884, 290885, 290886, 290915, 290972], "score": 0.913371, "input_data_size": 635435249, "oldest_snapshot_seqno": 4426433088} 2024/11/24-03:53:14.485242 139957215163968 (Original Log Time 2024/11/24-03:53:14.483681) [compaction_job.cc:2022] [default] [JOB 41096] Compacted 1@2 + 17@3 files to L3 => 610910049 bytes 2024/11/24-03:53:14.485243 139957215163968 (Original Log Time 2024/11/24-03:53:14.485161) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 18 0 0 0] max score 0.02, MB/sec: 11.1 rd, 10.7 wr, level 3, files in(1, 17) out(18 +0 blob) MB in(27.1, 578.9 +0.0 blob) out(582.6 +0.0 blob), read-write-amplify(43.9) write-amplify(21.5) OK, records in: 15638208, records dropped: 112340 output_compression: Snappy 2024/11/24-03:53:14.485249 139957215163968 (Original Log Time 2024/11/24-03:53:14.485206) EVENT_LOG_v1 {"time_micros": 1732391594485186, "cf": "default", "job": 41096, "event": "compaction_finished", "compaction_time_micros": 57300861, "compaction_time_cpu_micros": 23788849, "output_level": 3, "num_output_files": 18, "total_output_size": 610910049, "num_input_records": 15638208, "num_output_records": 15525868, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 431639003, "file_range_sync_nanos": 0, "file_fsync_nanos": 12458, "file_prepare_write_nanos": 44933, "lsm_state": [0, 0, 0, 18, 0, 0, 0]} 2024/11/24-03:53:14.608195 139957215163968 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732391536,"executesMs":57557,"compactionJobId":41096,"attempt":0,"compactionInputRawBytes":2268635152,"compactionInputZipBytes":635435249,"compactionOutputRawBytes":0,"compactionOutputZipBytes":610910049,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-517410e2c5930b35447644225c07814a-5203e67eee82600a-00"}}, response = 2024/11/24-03:53:14.608303 139957215163968 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41096/att-00: olev 3, work 57.557 s, result 7.771 ms, install 1.585 ms, input{raw 2.113 GiB zip 605.998 MiB}