2024/11/24-08:39:13.556955 139957517301312 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00: fork to child time = 0.007889 sec 2024/11/24-08:39:13.557679 139957517301312 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00 2024/11/24-08:39:13.563992 139957517301312 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00 2024/11/24-08:39:13.564874 139957517301312 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-08:39:13.565117 139957517301312 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-08:39:13.620966 139957517301312 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-44088/att-00/MANIFEST-206347 2024/11/24-08:39:13.648493 139957517301312 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-44088/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 297847, last_sequence is 4517543792, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 297672 2024/11/24-08:39:13.648499 139957517301312 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-08:39:13.648633 139957517301312 [version_set.cc:5721] Creating manifest 297847 2024/11/24-08:39:13.692216 139957517301312 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-08:39:13.718212 139957517301312 [compaction_job.cc:2437] [default] [JOB 44088] Compacting 1@2 + 30@3 files to L3, score 0.81, subcompactions 1 : 1 2024/11/24-08:39:13.718233 139957517301312 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 33049 Base level 2, inputs: [297808(27MB)], [297754(33MB) 297755(35MB) 297756(33MB) 297757(35MB) 297758(33MB) 297759(35MB) 297760(31MB) 297761(30MB) 297762(34MB) 297763(33MB) 297764(31MB) 297765(33MB) 297766(33MB) 297767(33MB) 297768(34MB) 297769(33MB) 297770(33MB) 297771(35MB) 297772(33MB) 297773(32MB) 297774(35MB) 297775(33MB) 297776(32MB) 297777(33MB) 297778(33MB) 297779(34MB) 297780(4835KB) 297731(33MB) 297732(33MB) 297839(35MB)] 2024/11/24-08:39:13.724875 139957517301312 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-08:39:13.724898 139957517301312 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-08:39:13.724913 139957517301312 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-08:39:13.724917 139957517301312 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-08:39:13.724927 139957517301312 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-08:39:14.376176 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297848.sst) /dev/shm/ToplingTemp/Topling-307465-7pS2Ae 2024/11/24-08:39:14.933002 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297848.sst): first pass time = 0.56's, 241.060'MB/sec 2024/11/24-08:39:16.412650 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297848.sst): old prealloc_size = 147639500, real_size = 31110768 2024/11/24-08:39:16.413248 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297848.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922275 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.53 avg-zval = 31.57 2024/11/24-08:39:16.414068 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297848.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:17.239595 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297848.sst): second pass time = 1.48's, 73.301'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.422'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3069.816'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 271.496'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30380.213'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.301'MB/sec zip pipeline throughput = 73.301'MB/sec entries = 922275 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.53 avg-zval = 31.57 usrkeys = 922275 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.53 avg-zval = 31.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922275 bytes = 3112688 } seq expand size = 6455925 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4473 value = 3.7230 dict = 1.00 all = 3.8260 } Zip/UnZip{ index = 0.0957 value = 0.2686 dict = 1.00 all = 0.2614 } ---------------------------- total value len = 2251.440423 GB avg = 0.200 KB (by entry num) total key len = 306.908786 GB avg = 0.027 KB total ukey len = 237.373173 GB avg = 0.015 KB total ukey num = 15.739583438 Billion total entry num = 11.280897386 Billion write speed all = 29.941464831 MB/sec (with seq num) write speed all = 28.885263149 MB/sec (without seq num) 2024/11/24-08:39:17.284708 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297848: 922275 keys, 35080208 bytes, temperature: kWarm 2024/11/24-08:39:17.284799 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408757284744, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297848, "file_size": 35080208, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4385885104, "largest_seqno": 4517407047, "table_properties": {"data_size": 29114464, "index_size": 1765568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25823700, "raw_average_key_size": 28, "raw_value_size": 108394030, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922275, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408754, "oldest_key_time": 0, "file_creation_time": 1732408754, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297848, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:17.321211 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297849.sst) /dev/shm/ToplingTemp/Topling-307465-H8Qsuf 2024/11/24-08:39:18.250593 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297849.sst): first pass time = 0.93's, 144.423'MB/sec 2024/11/24-08:39:19.751202 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297849.sst): old prealloc_size = 147639500, real_size = 33020448 2024/11/24-08:39:19.751888 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297849.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925672 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.99 avg-zval = 33.52 2024/11/24-08:39:19.752747 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297849.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:20.634676 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297849.sst): second pass time = 1.50's, 72.216'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.011'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2659.528'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 283.644'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34379.388'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.216'MB/sec zip pipeline throughput = 72.216'MB/sec entries = 925672 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.99 avg-zval = 33.52 usrkeys = 925672 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.99 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925672 bytes = 3239872 } seq expand size = 6479704 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5110 value = 3.4904 dict = 1.00 all = 3.6163 } Zip/UnZip{ index = 0.0951 value = 0.2865 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 2251.555202 GB avg = 0.200 KB (by entry num) total key len = 306.934705 GB avg = 0.027 KB total ukey len = 237.391686 GB avg = 0.015 KB total ukey num = 15.740509110 Billion total entry num = 11.281823058 Billion write speed all = 29.941921760 MB/sec (with seq num) write speed all = 28.885675378 MB/sec (without seq num) 2024/11/24-08:39:20.680238 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297849: 925672 keys, 37114870 bytes, temperature: kWarm 2024/11/24-08:39:20.680308 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408760680273, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297849, "file_size": 37114870, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4517406656, "table_properties": {"data_size": 31027472, "index_size": 1761344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25918816, "raw_average_key_size": 28, "raw_value_size": 108298956, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925672, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408757, "oldest_key_time": 0, "file_creation_time": 1732408757, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297849, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:20.715910 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297850.sst) /dev/shm/ToplingTemp/Topling-307465-T2BkGh 2024/11/24-08:39:21.752865 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297850.sst): first pass time = 1.04's, 129.439'MB/sec 2024/11/24-08:39:23.128670 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297850.sst): old prealloc_size = 147639500, real_size = 31176880 2024/11/24-08:39:23.129287 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297850.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921553 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.64 avg-zval = 31.65 2024/11/24-08:39:23.130065 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297850.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:23.940893 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297850.sst): second pass time = 1.37's, 78.875'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.413'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2986.278'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 280.617'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 38012.558'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.875'MB/sec zip pipeline throughput = 78.875'MB/sec entries = 921553 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.64 avg-zval = 31.65 usrkeys = 921553 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.64 avg-zval = 31.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921553 bytes = 3225456 } seq expand size = 6450871 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.3797 value = 3.7166 dict = 1.00 all = 3.8066 } Zip/UnZip{ index = 0.0963 value = 0.2691 dict = 1.00 all = 0.2627 } ---------------------------- total value len = 2251.670067 GB avg = 0.200 KB (by entry num) total key len = 306.960508 GB avg = 0.027 KB total ukey len = 237.410117 GB avg = 0.015 KB total ukey num = 15.741430663 Billion total entry num = 11.282744611 Billion write speed all = 29.942409456 MB/sec (with seq num) write speed all = 28.886117666 MB/sec (without seq num) 2024/11/24-08:39:23.986154 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297850: 921553 keys, 35259678 bytes, temperature: kWarm 2024/11/24-08:39:23.986228 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408763986189, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297850, "file_size": 35259678, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4382051628, "largest_seqno": 4517407061, "table_properties": {"data_size": 29170656, "index_size": 1775680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25803484, "raw_average_key_size": 28, "raw_value_size": 108414346, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921553, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408760, "oldest_key_time": 0, "file_creation_time": 1732408760, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297850, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:24.024259 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297851.sst) /dev/shm/ToplingTemp/Topling-307465-nf0kji 2024/11/24-08:39:24.919658 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297851.sst): first pass time = 0.90's, 149.903'MB/sec 2024/11/24-08:39:26.264451 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297851.sst): old prealloc_size = 147639500, real_size = 33062000 2024/11/24-08:39:26.265036 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297851.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926018 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.94 avg-zval = 33.55 2024/11/24-08:39:26.265788 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297851.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:27.114697 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297851.sst): second pass time = 1.34's, 80.601'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.703'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3133.502'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 290.879'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32396.089'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.601'MB/sec zip pipeline throughput = 80.601'MB/sec entries = 926018 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 33.55 usrkeys = 926018 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.94 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926018 bytes = 3241072 } seq expand size = 6482126 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5138 value = 3.4855 dict = 1.00 all = 3.6121 } Zip/UnZip{ index = 0.0951 value = 0.2869 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 2251.784838 GB avg = 0.200 KB (by entry num) total key len = 306.986437 GB avg = 0.027 KB total ukey len = 237.428638 GB avg = 0.015 KB total ukey num = 15.742356681 Billion total entry num = 11.283670629 Billion write speed all = 29.942943871 MB/sec (with seq num) write speed all = 28.886604621 MB/sec (without seq num) 2024/11/24-08:39:27.161621 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297851: 926018 keys, 37157672 bytes, temperature: kWarm 2024/11/24-08:39:27.161693 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408767161658, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297851, "file_size": 37157672, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4517407236, "table_properties": {"data_size": 31068768, "index_size": 1761536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928504, "raw_average_key_size": 28, "raw_value_size": 108289284, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926018, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408764, "oldest_key_time": 0, "file_creation_time": 1732408764, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297851, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:27.203123 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297852.sst) /dev/shm/ToplingTemp/Topling-307465-G5105g 2024/11/24-08:39:28.029181 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297852.sst): first pass time = 0.83's, 162.487'MB/sec 2024/11/24-08:39:29.336903 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297852.sst): old prealloc_size = 147639500, real_size = 31248352 2024/11/24-08:39:29.337470 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297852.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921203 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.70 avg-zval = 31.76 2024/11/24-08:39:29.338270 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297852.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:30.147745 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297852.sst): second pass time = 1.31's, 82.959'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.573'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3241.510'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.12's, 284.881'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 36286.547'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.959'MB/sec zip pipeline throughput = 82.959'MB/sec entries = 921203 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.70 avg-zval = 31.76 usrkeys = 921203 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.70 avg-zval = 31.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921203 bytes = 3224224 } seq expand size = 6448421 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4447 value = 3.7063 dict = 1.00 all = 3.7990 } Zip/UnZip{ index = 0.0957 value = 0.2698 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 2251.899711 GB avg = 0.200 KB (by entry num) total key len = 307.012231 GB avg = 0.027 KB total ukey len = 237.447062 GB avg = 0.015 KB total ukey num = 15.743277884 Billion total entry num = 11.284591832 Billion write speed all = 29.943527175 MB/sec (with seq num) write speed all = 28.887139179 MB/sec (without seq num) 2024/11/24-08:39:30.191249 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297852: 921203 keys, 35329958 bytes, temperature: kWarm 2024/11/24-08:39:30.191320 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408770191284, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297852, "file_size": 35329958, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4379564543, "largest_seqno": 4517407073, "table_properties": {"data_size": 29253904, "index_size": 1763968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25793684, "raw_average_key_size": 28, "raw_value_size": 108424158, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921203, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408767, "oldest_key_time": 0, "file_creation_time": 1732408767, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297852, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:30.228762 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297853.sst) /dev/shm/ToplingTemp/Topling-307465-mTIMfe 2024/11/24-08:39:31.137387 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297853.sst): first pass time = 0.91's, 147.722'MB/sec 2024/11/24-08:39:32.477728 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297853.sst): old prealloc_size = 147639500, real_size = 33327344 2024/11/24-08:39:32.478352 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297853.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926788 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.82 avg-zval = 33.80 2024/11/24-08:39:32.479131 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297853.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:33.327928 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297853.sst): second pass time = 1.34's, 80.822'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.025'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2951.981'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 299.219'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32771.537'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.822'MB/sec zip pipeline throughput = 80.822'MB/sec entries = 926788 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.82 avg-zval = 33.80 usrkeys = 926788 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.82 avg-zval = 33.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926788 bytes = 3243776 } seq expand size = 6487516 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4451 value = 3.4567 dict = 1.00 all = 3.5859 } Zip/UnZip{ index = 0.0957 value = 0.2893 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 2252.014466 GB avg = 0.200 KB (by entry num) total key len = 307.038181 GB avg = 0.027 KB total ukey len = 237.465598 GB avg = 0.015 KB total ukey num = 15.744204672 Billion total entry num = 11.285518620 Billion write speed all = 29.944059339 MB/sec (with seq num) write speed all = 28.887623897 MB/sec (without seq num) 2024/11/24-08:39:33.374228 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297853: 926788 keys, 37429058 bytes, temperature: kWarm 2024/11/24-08:39:33.374302 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408773374262, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297853, "file_size": 37429058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4517407504, "table_properties": {"data_size": 31320864, "index_size": 1774592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950064, "raw_average_key_size": 28, "raw_value_size": 108267724, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926788, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408770, "oldest_key_time": 0, "file_creation_time": 1732408770, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297853, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:33.408551 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297854.sst) /dev/shm/ToplingTemp/Topling-307465-ricMef 2024/11/24-08:39:34.247702 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297854.sst): first pass time = 0.84's, 159.951'MB/sec 2024/11/24-08:39:35.526784 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297854.sst): old prealloc_size = 147639500, real_size = 28954976 2024/11/24-08:39:35.527406 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297854.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 938670 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1079 GB zip-val = 0.0269 GB avg-val = 114.99 avg-zval = 28.68 2024/11/24-08:39:35.528217 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297854.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:36.352471 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297854.sst): second pass time = 1.28's, 84.442'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 40.073'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3012.662'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.336'MB/sec write SST data time = 0.13's, 261.986'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 32512.090'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.442'MB/sec zip pipeline throughput = 84.442'MB/sec entries = 938670 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.99 avg-zval = 28.68 usrkeys = 938670 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.99 avg-zval = 28.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 938670 bytes = 3285360 } seq expand size = 6570690 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0269 GB dict = 1.08 MB all = 0.0331 GB } UnZip/Zip{ index = 10.4438 value = 4.0091 dict = 1.00 all = 4.0565 } Zip/UnZip{ index = 0.0958 value = 0.2494 dict = 1.00 all = 0.2465 } ---------------------------- total value len = 2252.128972 GB avg = 0.200 KB (by entry num) total key len = 307.064463 GB avg = 0.027 KB total ukey len = 237.484371 GB avg = 0.015 KB total ukey num = 15.745143342 Billion total entry num = 11.286457290 Billion write speed all = 29.944646971 MB/sec (with seq num) write speed all = 28.888161050 MB/sec (without seq num) 2024/11/24-08:39:36.392836 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297854: 938670 keys, 33086708 bytes, temperature: kWarm 2024/11/24-08:39:36.392913 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408776392878, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297854, "file_size": 33086708, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4517405992, "table_properties": {"data_size": 26922576, "index_size": 1797568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26282760, "raw_average_key_size": 28, "raw_value_size": 107935092, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 938670, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408773, "oldest_key_time": 0, "file_creation_time": 1732408773, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297854, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:36.430433 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297855.sst) /dev/shm/ToplingTemp/Topling-307465-oqWhVf 2024/11/24-08:39:37.289069 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297855.sst): first pass time = 0.86's, 156.322'MB/sec 2024/11/24-08:39:38.548336 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297855.sst): old prealloc_size = 147639500, real_size = 27764112 2024/11/24-08:39:38.548880 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297855.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.21 2024/11/24-08:39:38.549677 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297855.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:39.380048 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297855.sst): second pass time = 1.26's, 85.653'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 38.231'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3466.949'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.335'MB/sec write SST data time = 0.13's, 252.987'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 29861.296'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.653'MB/sec zip pipeline throughput = 85.653'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.21 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.4461 value = 4.1898 dict = 1.00 all = 4.2213 } Zip/UnZip{ index = 0.0957 value = 0.2387 dict = 1.00 all = 0.2369 } ---------------------------- total value len = 2252.243341 GB avg = 0.200 KB (by entry num) total key len = 307.090929 GB avg = 0.027 KB total ukey len = 237.503275 GB avg = 0.015 KB total ukey num = 15.746088538 Billion total entry num = 11.287402486 Billion write speed all = 29.945234040 MB/sec (with seq num) write speed all = 28.888697071 MB/sec (without seq num) 2024/11/24-08:39:39.420815 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297855: 945196 keys, 31795312 bytes, temperature: kWarm 2024/11/24-08:39:39.420894 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408779420849, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297855, "file_size": 31795312, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4340137702, "largest_seqno": 4442605153, "table_properties": {"data_size": 25717952, "index_size": 1809664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408776, "oldest_key_time": 0, "file_creation_time": 1732408776, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297855, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:39.456733 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297856.sst) /dev/shm/ToplingTemp/Topling-307465-kiW0be 2024/11/24-08:39:40.467122 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297856.sst): first pass time = 1.01's, 132.843'MB/sec 2024/11/24-08:39:41.791060 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297856.sst): old prealloc_size = 147639500, real_size = 31575504 2024/11/24-08:39:41.791604 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297856.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 937496 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0296 GB avg-val = 115.17 avg-zval = 31.53 2024/11/24-08:39:41.792454 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297856.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:42.651116 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297856.sst): second pass time = 1.32's, 81.600'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.516'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3487.315'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 276.809'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 32360.357'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.600'MB/sec zip pipeline throughput = 81.600'MB/sec entries = 937496 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.17 avg-zval = 31.53 usrkeys = 937496 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.17 avg-zval = 31.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 937496 bytes = 3281248 } seq expand size = 6562472 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5116 value = 3.6528 dict = 1.00 all = 3.7591 } Zip/UnZip{ index = 0.0951 value = 0.2738 dict = 1.00 all = 0.2660 } ---------------------------- total value len = 2252.357871 GB avg = 0.200 KB (by entry num) total key len = 307.117179 GB avg = 0.027 KB total ukey len = 237.522025 GB avg = 0.015 KB total ukey num = 15.747026034 Billion total entry num = 11.288339982 Billion write speed all = 29.945735111 MB/sec (with seq num) write speed all = 28.889150828 MB/sec (without seq num) 2024/11/24-08:39:42.698558 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297856: 937496 keys, 35704772 bytes, temperature: kWarm 2024/11/24-08:39:42.698630 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408782698593, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297856, "file_size": 35704772, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4517405601, "table_properties": {"data_size": 29557184, "index_size": 1783744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26249888, "raw_average_key_size": 28, "raw_value_size": 107967880, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 937496, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408779, "oldest_key_time": 0, "file_creation_time": 1732408779, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297856, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:42.744044 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297857.sst) /dev/shm/ToplingTemp/Topling-307465-N7kDig 2024/11/24-08:39:43.636260 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297857.sst): first pass time = 0.89's, 150.439'MB/sec 2024/11/24-08:39:44.949669 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297857.sst): old prealloc_size = 147639500, real_size = 31576432 2024/11/24-08:39:44.950260 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297857.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0296 GB avg-val = 118.00 avg-zval = 32.17 2024/11/24-08:39:44.951051 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297857.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:45.772211 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297857.sst): second pass time = 1.31's, 82.643'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.347'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3130.489'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 285.054'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 25912.543'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.643'MB/sec zip pipeline throughput = 82.643'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 32.17 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 32.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.09 MB all = 0.0357 GB } UnZip/Zip{ index = 10.3764 value = 3.6679 dict = 1.00 all = 3.7645 } Zip/UnZip{ index = 0.0964 value = 0.2726 dict = 1.00 all = 0.2656 } ---------------------------- total value len = 2252.472784 GB avg = 0.200 KB (by entry num) total key len = 307.142919 GB avg = 0.027 KB total ukey len = 237.540411 GB avg = 0.015 KB total ukey num = 15.747945334 Billion total entry num = 11.289259282 Billion write speed all = 29.946287205 MB/sec (with seq num) write speed all = 28.889655460 MB/sec (without seq num) 2024/11/24-08:39:45.817718 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297857: 919300 keys, 35653170 bytes, temperature: kWarm 2024/11/24-08:39:45.817789 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408785817753, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297857, "file_size": 35653170, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4306050411, "largest_seqno": 4453863069, "table_properties": {"data_size": 29574496, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408782, "oldest_key_time": 0, "file_creation_time": 1732408782, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297857, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:45.912067 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297858.sst) /dev/shm/ToplingTemp/Topling-307465-fDni7e 2024/11/24-08:39:46.739759 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297858.sst): first pass time = 0.83's, 162.168'MB/sec 2024/11/24-08:39:48.030846 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297858.sst): old prealloc_size = 147639500, real_size = 29447968 2024/11/24-08:39:48.031449 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297858.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935485 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0274 GB avg-val = 115.47 avg-zval = 29.30 2024/11/24-08:39:48.032270 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297858.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:48.855758 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297858.sst): second pass time = 1.29's, 83.745'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 40.697'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3190.070'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.336'MB/sec write SST data time = 0.13's, 262.480'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30736.434'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.745'MB/sec zip pipeline throughput = 83.745'MB/sec entries = 935485 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.47 avg-zval = 29.30 usrkeys = 935485 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.47 avg-zval = 29.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935485 bytes = 3274208 } seq expand size = 6548395 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0274 GB dict = 1.08 MB all = 0.0336 GB } UnZip/Zip{ index = 10.3766 value = 3.9409 dict = 1.00 all = 3.9981 } Zip/UnZip{ index = 0.0964 value = 0.2537 dict = 1.00 all = 0.2501 } ---------------------------- total value len = 2252.587356 GB avg = 0.200 KB (by entry num) total key len = 307.169113 GB avg = 0.027 KB total ukey len = 237.559120 GB avg = 0.015 KB total ukey num = 15.748880819 Billion total entry num = 11.290194767 Billion write speed all = 29.946853739 MB/sec (with seq num) write speed all = 28.890172558 MB/sec (without seq num) 2024/11/24-08:39:48.900379 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297858: 935485 keys, 33570136 bytes, temperature: kWarm 2024/11/24-08:39:48.900453 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408788900417, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297858, "file_size": 33570136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4370278847, "largest_seqno": 4517406789, "table_properties": {"data_size": 27410832, "index_size": 1803072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26193580, "raw_average_key_size": 28, "raw_value_size": 108024270, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935485, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408785, "oldest_key_time": 0, "file_creation_time": 1732408785, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297858, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:48.940930 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297859.sst) /dev/shm/ToplingTemp/Topling-307465-RBAYri 2024/11/24-08:39:49.960722 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297859.sst): first pass time = 1.02's, 131.619'MB/sec 2024/11/24-08:39:51.257491 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297859.sst): old prealloc_size = 147639500, real_size = 30676448 2024/11/24-08:39:51.258081 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297859.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 939616 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1079 GB zip-val = 0.0287 GB avg-val = 114.84 avg-zval = 30.50 2024/11/24-08:39:51.258894 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297859.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:52.105945 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297859.sst): second pass time = 1.30's, 83.263'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.032'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3159.967'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.326'MB/sec write SST data time = 0.13's, 273.767'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 33855.777'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.263'MB/sec zip pipeline throughput = 83.263'MB/sec entries = 939616 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.84 avg-zval = 30.50 usrkeys = 939616 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.84 avg-zval = 30.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 939616 bytes = 3288672 } seq expand size = 6577312 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.5142 value = 3.7659 dict = 1.00 all = 3.8554 } Zip/UnZip{ index = 0.0951 value = 0.2655 dict = 1.00 all = 0.2594 } ---------------------------- total value len = 2252.701842 GB avg = 0.200 KB (by entry num) total key len = 307.195422 GB avg = 0.027 KB total ukey len = 237.577913 GB avg = 0.015 KB total ukey num = 15.749820435 Billion total entry num = 11.291134383 Billion write speed all = 29.947362186 MB/sec (with seq num) write speed all = 28.890633244 MB/sec (without seq num) 2024/11/24-08:39:52.149336 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297859: 939616 keys, 34812626 bytes, temperature: kWarm 2024/11/24-08:39:52.149413 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408792149376, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297859, "file_size": 34812626, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4517405152, "table_properties": {"data_size": 28654032, "index_size": 1787328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26309248, "raw_average_key_size": 28, "raw_value_size": 107908488, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 939616, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408788, "oldest_key_time": 0, "file_creation_time": 1732408788, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297859, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:52.188309 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297860.sst) /dev/shm/ToplingTemp/Topling-307465-4tcgUh 2024/11/24-08:39:53.199619 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297860.sst): first pass time = 1.01's, 132.723'MB/sec 2024/11/24-08:39:54.485778 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297860.sst): old prealloc_size = 147639500, real_size = 31309168 2024/11/24-08:39:54.486353 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297860.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.89 2024/11/24-08:39:54.487097 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297860.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:55.299248 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297860.sst): second pass time = 1.29's, 84.396'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.360'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.668'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 277.542'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 26756.017'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.396'MB/sec zip pipeline throughput = 84.396'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.89 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6999 dict = 1.00 all = 3.8053 } Zip/UnZip{ index = 0.0957 value = 0.2703 dict = 1.00 all = 0.2628 } ---------------------------- total value len = 2252.816755 GB avg = 0.200 KB (by entry num) total key len = 307.221162 GB avg = 0.027 KB total ukey len = 237.596299 GB avg = 0.015 KB total ukey num = 15.750739735 Billion total entry num = 11.292053683 Billion write speed all = 29.947888864 MB/sec (with seq num) write speed all = 28.891113364 MB/sec (without seq num) 2024/11/24-08:39:55.342839 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297860: 919300 keys, 35271026 bytes, temperature: kWarm 2024/11/24-08:39:55.342913 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408795342877, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297860, "file_size": 35271026, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4289322836, "largest_seqno": 4407285315, "table_properties": {"data_size": 29318944, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408792, "oldest_key_time": 0, "file_creation_time": 1732408792, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297860, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:55.380215 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297861.sst) /dev/shm/ToplingTemp/Topling-307465-7hqX6g 2024/11/24-08:39:56.341281 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297861.sst): first pass time = 0.96's, 139.661'MB/sec 2024/11/24-08:39:57.652971 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297861.sst): old prealloc_size = 147639500, real_size = 31169456 2024/11/24-08:39:57.653553 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.48 avg-zval = 31.62 2024/11/24-08:39:57.654338 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297861.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:39:58.465742 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297861.sst): second pass time = 1.31's, 82.706'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.236'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3277.638'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 275.661'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32479.400'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.706'MB/sec zip pipeline throughput = 82.706'MB/sec entries = 922568 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.48 avg-zval = 31.62 usrkeys = 922568 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.48 avg-zval = 31.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922568 bytes = 3113680 } seq expand size = 6457976 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4420 value = 3.7155 dict = 1.00 all = 3.8195 } Zip/UnZip{ index = 0.0958 value = 0.2691 dict = 1.00 all = 0.2618 } ---------------------------- total value len = 2252.931598 GB avg = 0.199 KB (by entry num) total key len = 307.246994 GB avg = 0.027 KB total ukey len = 237.614750 GB avg = 0.015 KB total ukey num = 15.751662303 Billion total entry num = 11.292976251 Billion write speed all = 29.948425161 MB/sec (with seq num) write speed all = 28.891602469 MB/sec (without seq num) 2024/11/24-08:39:58.509326 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297861: 922568 keys, 35139716 bytes, temperature: kWarm 2024/11/24-08:39:58.509397 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408798509361, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297861, "file_size": 35139716, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4407285316, "largest_seqno": 4517406800, "table_properties": {"data_size": 29171616, "index_size": 1767040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25831904, "raw_average_key_size": 28, "raw_value_size": 108385896, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408795, "oldest_key_time": 0, "file_creation_time": 1732408795, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:39:58.545135 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297862.sst) /dev/shm/ToplingTemp/Topling-307465-VrDHGg 2024/11/24-08:39:59.560550 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297862.sst): first pass time = 1.02's, 132.186'MB/sec 2024/11/24-08:40:00.871194 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297862.sst): old prealloc_size = 147639500, real_size = 31819712 2024/11/24-08:40:00.871724 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936398 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0298 GB avg-val = 115.33 avg-zval = 31.83 2024/11/24-08:40:00.872548 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297862.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:01.734942 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297862.sst): second pass time = 1.31's, 82.456'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.617'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3497.634'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.318'MB/sec write SST data time = 0.13's, 280.418'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 30552.393'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.456'MB/sec zip pipeline throughput = 82.456'MB/sec entries = 936398 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.33 avg-zval = 31.83 usrkeys = 936398 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.33 avg-zval = 31.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936398 bytes = 3277408 } seq expand size = 6554786 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5143 value = 3.6236 dict = 1.00 all = 3.7339 } Zip/UnZip{ index = 0.0951 value = 0.2760 dict = 1.00 all = 0.2678 } ---------------------------- total value len = 2253.046152 GB avg = 0.199 KB (by entry num) total key len = 307.273214 GB avg = 0.027 KB total ukey len = 237.633478 GB avg = 0.015 KB total ukey num = 15.752598701 Billion total entry num = 11.293912649 Billion write speed all = 29.948926569 MB/sec (with seq num) write speed all = 28.892056664 MB/sec (without seq num) 2024/11/24-08:40:01.782067 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297862: 936398 keys, 35945702 bytes, temperature: kWarm 2024/11/24-08:40:01.782142 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408801782103, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297862, "file_size": 35945702, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4517406381, "table_properties": {"data_size": 29804272, "index_size": 1781184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26219144, "raw_average_key_size": 28, "raw_value_size": 107998700, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936398, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408798, "oldest_key_time": 0, "file_creation_time": 1732408798, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:01.819049 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297863.sst) /dev/shm/ToplingTemp/Topling-307465-SM58mh 2024/11/24-08:40:02.669764 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297863.sst): first pass time = 0.85's, 157.780'MB/sec 2024/11/24-08:40:03.983178 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297863.sst): old prealloc_size = 147639500, real_size = 31375152 2024/11/24-08:40:03.983710 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297863.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.96 2024/11/24-08:40:03.984480 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297863.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:04.795174 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297863.sst): second pass time = 1.31's, 82.674'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.520'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3460.054'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 281.463'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33934.134'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.674'MB/sec zip pipeline throughput = 82.674'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.96 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4447 value = 3.6916 dict = 1.00 all = 3.7982 } Zip/UnZip{ index = 0.0957 value = 0.2709 dict = 1.00 all = 0.2633 } ---------------------------- total value len = 2253.161064 GB avg = 0.199 KB (by entry num) total key len = 307.298954 GB avg = 0.027 KB total ukey len = 237.651864 GB avg = 0.015 KB total ukey num = 15.753518001 Billion total entry num = 11.294831949 Billion write speed all = 29.949499746 MB/sec (with seq num) write speed all = 28.892581648 MB/sec (without seq num) 2024/11/24-08:40:04.838496 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297863: 919300 keys, 35337010 bytes, temperature: kWarm 2024/11/24-08:40:04.838570 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408804838534, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297863, "file_size": 35337010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4301741383, "largest_seqno": 4413109231, "table_properties": {"data_size": 29384800, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408801, "oldest_key_time": 0, "file_creation_time": 1732408801, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297863, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:04.873893 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297864.sst) /dev/shm/ToplingTemp/Topling-307465-05CCFf 2024/11/24-08:40:05.706667 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297864.sst): first pass time = 0.83's, 161.177'MB/sec 2024/11/24-08:40:07.002691 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297864.sst): old prealloc_size = 147639500, real_size = 31266272 2024/11/24-08:40:07.003281 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297864.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921775 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.61 avg-zval = 31.75 2024/11/24-08:40:07.004045 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297864.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:07.819737 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297864.sst): second pass time = 1.29's, 83.738'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.126'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3109.414'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 279.423'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33931.177'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.738'MB/sec zip pipeline throughput = 83.738'MB/sec entries = 921775 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.61 avg-zval = 31.75 usrkeys = 921775 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.61 avg-zval = 31.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921775 bytes = 3111008 } seq expand size = 6452425 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4447 value = 3.7037 dict = 1.00 all = 3.8093 } Zip/UnZip{ index = 0.0957 value = 0.2700 dict = 1.00 all = 0.2625 } ---------------------------- total value len = 2253.275925 GB avg = 0.199 KB (by entry num) total key len = 307.324764 GB avg = 0.027 KB total ukey len = 237.670300 GB avg = 0.015 KB total ukey num = 15.754439776 Billion total entry num = 11.295753724 Billion write speed all = 29.950085578 MB/sec (with seq num) write speed all = 28.893118618 MB/sec (without seq num) 2024/11/24-08:40:07.862582 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297864: 921775 keys, 35234460 bytes, temperature: kWarm 2024/11/24-08:40:07.862647 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408807862611, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297864, "file_size": 35234460, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4413109232, "largest_seqno": 4517406820, "table_properties": {"data_size": 29270608, "index_size": 1765056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25809700, "raw_average_key_size": 28, "raw_value_size": 108408134, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921775, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408804, "oldest_key_time": 0, "file_creation_time": 1732408804, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297864, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:07.899379 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297865.sst) /dev/shm/ToplingTemp/Topling-307465-h3Y6mi 2024/11/24-08:40:08.857504 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297865.sst): first pass time = 0.96's, 140.089'MB/sec 2024/11/24-08:40:10.205517 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297865.sst): old prealloc_size = 147639500, real_size = 32693984 2024/11/24-08:40:10.206105 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297865.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932066 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 116.00 avg-zval = 32.92 2024/11/24-08:40:10.206884 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297865.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:11.068747 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297865.sst): second pass time = 1.35's, 80.258'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.640'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3141.377'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.13's, 292.862'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35496.986'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.258'MB/sec zip pipeline throughput = 80.258'MB/sec entries = 932066 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.00 avg-zval = 32.92 usrkeys = 932066 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.00 avg-zval = 32.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932066 bytes = 3262240 } seq expand size = 6524462 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5137 value = 3.5232 dict = 1.00 all = 3.6466 } Zip/UnZip{ index = 0.0951 value = 0.2838 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 2253.390569 GB avg = 0.199 KB (by entry num) total key len = 307.350861 GB avg = 0.027 KB total ukey len = 237.688941 GB avg = 0.015 KB total ukey num = 15.755371842 Billion total entry num = 11.296685790 Billion write speed all = 29.950593586 MB/sec (with seq num) write speed all = 28.893579579 MB/sec (without seq num) 2024/11/24-08:40:11.114993 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297865: 932066 keys, 36806700 bytes, temperature: kWarm 2024/11/24-08:40:11.115074 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408811115036, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297865, "file_size": 36806700, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4517405691, "table_properties": {"data_size": 30687696, "index_size": 1773056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26097848, "raw_average_key_size": 28, "raw_value_size": 108119952, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 932066, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408807, "oldest_key_time": 0, "file_creation_time": 1732408807, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297865, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:11.150735 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297866.sst) /dev/shm/ToplingTemp/Topling-307465-DjUGxf 2024/11/24-08:40:12.159116 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297866.sst): first pass time = 1.01's, 133.107'MB/sec 2024/11/24-08:40:13.444245 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297866.sst): old prealloc_size = 147639500, real_size = 31237856 2024/11/24-08:40:13.444765 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297866.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0292 GB avg-val = 118.00 avg-zval = 31.82 2024/11/24-08:40:13.445545 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297866.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:14.257566 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297866.sst): second pass time = 1.28's, 84.466'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.280'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3560.785'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 279.432'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34176.773'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.466'MB/sec zip pipeline throughput = 84.466'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.82 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.09 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4458 value = 3.7089 dict = 1.00 all = 3.8130 } Zip/UnZip{ index = 0.0957 value = 0.2696 dict = 1.00 all = 0.2623 } ---------------------------- total value len = 2253.505482 GB avg = 0.199 KB (by entry num) total key len = 307.376602 GB avg = 0.027 KB total ukey len = 237.707327 GB avg = 0.015 KB total ukey num = 15.756291142 Billion total entry num = 11.297605090 Billion write speed all = 29.951121595 MB/sec (with seq num) write speed all = 28.894060995 MB/sec (without seq num) 2024/11/24-08:40:14.300868 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297866: 919300 keys, 35199666 bytes, temperature: kWarm 2024/11/24-08:40:14.300941 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408814300903, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297866, "file_size": 35199666, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4319448361, "largest_seqno": 4430381173, "table_properties": {"data_size": 29247696, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408811, "oldest_key_time": 0, "file_creation_time": 1732408811, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297866, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:14.347436 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297867.sst) /dev/shm/ToplingTemp/Topling-307465-Nd8mlf 2024/11/24-08:40:15.349624 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297867.sst): first pass time = 1.00's, 133.930'MB/sec 2024/11/24-08:40:16.658073 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297867.sst): old prealloc_size = 147639500, real_size = 30803472 2024/11/24-08:40:16.658639 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297867.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925434 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 117.03 avg-zval = 31.12 2024/11/24-08:40:16.659395 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297867.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:17.475373 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297867.sst): second pass time = 1.31's, 82.823'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.557'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3252.052'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 274.640'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31947.678'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.823'MB/sec zip pipeline throughput = 82.823'MB/sec entries = 925434 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.03 avg-zval = 31.12 usrkeys = 925434 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.03 avg-zval = 31.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925434 bytes = 3123360 } seq expand size = 6478038 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4441 value = 3.7606 dict = 1.00 all = 3.8590 } Zip/UnZip{ index = 0.0957 value = 0.2659 dict = 1.00 all = 0.2591 } ---------------------------- total value len = 2253.620266 GB avg = 0.199 KB (by entry num) total key len = 307.402514 GB avg = 0.027 KB total ukey len = 237.725836 GB avg = 0.015 KB total ukey num = 15.757216576 Billion total entry num = 11.298530524 Billion write speed all = 29.951639914 MB/sec (with seq num) write speed all = 28.894532510 MB/sec (without seq num) 2024/11/24-08:40:17.518943 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297867: 925434 keys, 34780166 bytes, temperature: kWarm 2024/11/24-08:40:17.519034 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408817518993, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297867, "file_size": 34780166, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4430381174, "largest_seqno": 4517406833, "table_properties": {"data_size": 28799744, "index_size": 1772160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912152, "raw_average_key_size": 28, "raw_value_size": 108305672, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408814, "oldest_key_time": 0, "file_creation_time": 1732408814, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297867, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:17.556478 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297868.sst) /dev/shm/ToplingTemp/Topling-307465-xhvCVf 2024/11/24-08:40:18.587319 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297868.sst): first pass time = 1.03's, 130.207'MB/sec 2024/11/24-08:40:19.941293 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297868.sst): old prealloc_size = 147639500, real_size = 33044176 2024/11/24-08:40:19.941841 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297868.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927104 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.77 avg-zval = 33.49 2024/11/24-08:40:19.942659 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297868.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:20.792735 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297868.sst): second pass time = 1.35's, 79.999'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.625'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3258.524'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 294.157'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32452.830'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.999'MB/sec zip pipeline throughput = 79.999'MB/sec entries = 927104 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 33.49 usrkeys = 927104 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.77 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927104 bytes = 3244880 } seq expand size = 6489728 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5131 value = 3.4868 dict = 1.00 all = 3.6135 } Zip/UnZip{ index = 0.0951 value = 0.2868 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 2253.735014 GB avg = 0.199 KB (by entry num) total key len = 307.428473 GB avg = 0.027 KB total ukey len = 237.744378 GB avg = 0.015 KB total ukey num = 15.758143680 Billion total entry num = 11.299457628 Billion write speed all = 29.952123457 MB/sec (with seq num) write speed all = 28.894970326 MB/sec (without seq num) 2024/11/24-08:40:20.838092 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297868: 927104 keys, 37143280 bytes, temperature: kWarm 2024/11/24-08:40:20.838165 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408820838127, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297868, "file_size": 37143280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4517407129, "table_properties": {"data_size": 31048512, "index_size": 1763712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958912, "raw_average_key_size": 28, "raw_value_size": 108258932, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927104, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408817, "oldest_key_time": 0, "file_creation_time": 1732408817, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297868, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:20.875757 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297869.sst) /dev/shm/ToplingTemp/Topling-307465-rgyYSg 2024/11/24-08:40:21.807425 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297869.sst): first pass time = 0.93's, 144.070'MB/sec 2024/11/24-08:40:23.108104 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297869.sst): old prealloc_size = 147639500, real_size = 31393488 2024/11/24-08:40:23.108687 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297869.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.97 2024/11/24-08:40:23.109563 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297869.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:23.926773 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297869.sst): second pass time = 1.30's, 83.451'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.187'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3168.404'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 276.971'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33860.005'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.451'MB/sec zip pipeline throughput = 83.451'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.97 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.3779 value = 3.6907 dict = 1.00 all = 3.7963 } Zip/UnZip{ index = 0.0964 value = 0.2709 dict = 1.00 all = 0.2634 } ---------------------------- total value len = 2253.849927 GB avg = 0.199 KB (by entry num) total key len = 307.454213 GB avg = 0.027 KB total ukey len = 237.762764 GB avg = 0.015 KB total ukey num = 15.759062980 Billion total entry num = 11.300376928 Billion write speed all = 29.952670537 MB/sec (with seq num) write speed all = 28.895470147 MB/sec (without seq num) 2024/11/24-08:40:23.970234 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297869: 919300 keys, 35355186 bytes, temperature: kWarm 2024/11/24-08:40:23.970305 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408823970270, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297869, "file_size": 35355186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4340227706, "largest_seqno": 4452050417, "table_properties": {"data_size": 29391808, "index_size": 1771648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408820, "oldest_key_time": 0, "file_creation_time": 1732408820, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297869, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:24.005469 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297870.sst) /dev/shm/ToplingTemp/Topling-307465-7SsV0e 2024/11/24-08:40:24.923205 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297870.sst): first pass time = 0.92's, 146.255'MB/sec 2024/11/24-08:40:26.231804 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297870.sst): old prealloc_size = 147639500, real_size = 30267680 2024/11/24-08:40:26.232432 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297870.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930381 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0283 GB avg-val = 116.26 avg-zval = 30.37 2024/11/24-08:40:26.233186 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297870.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:27.053502 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297870.sst): second pass time = 1.31's, 82.711'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.551'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2955.753'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 267.438'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35737.932'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.711'MB/sec zip pipeline throughput = 82.711'MB/sec entries = 930381 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.26 avg-zval = 30.37 usrkeys = 930381 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.26 avg-zval = 30.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930381 bytes = 3023760 } seq expand size = 6512667 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0341 GB } UnZip/Zip{ index = 10.4479 value = 3.8284 dict = 1.00 all = 3.9312 } Zip/UnZip{ index = 0.0957 value = 0.2612 dict = 1.00 all = 0.2544 } ---------------------------- total value len = 2253.964607 GB avg = 0.199 KB (by entry num) total key len = 307.480264 GB avg = 0.027 KB total ukey len = 237.781371 GB avg = 0.015 KB total ukey num = 15.759993361 Billion total entry num = 11.301307309 Billion write speed all = 29.953221046 MB/sec (with seq num) write speed all = 28.895972273 MB/sec (without seq num) 2024/11/24-08:40:27.096203 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297870: 930381 keys, 34141478 bytes, temperature: kWarm 2024/11/24-08:40:27.096270 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408827096235, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297870, "file_size": 34141478, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4452050418, "largest_seqno": 4517406853, "table_properties": {"data_size": 28253904, "index_size": 1780992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26050668, "raw_average_key_size": 28, "raw_value_size": 108167130, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930381, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408823, "oldest_key_time": 0, "file_creation_time": 1732408823, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297870, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:27.134178 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297871.sst) /dev/shm/ToplingTemp/Topling-307465-1QMZii 2024/11/24-08:40:28.147635 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297871.sst): first pass time = 1.01's, 132.441'MB/sec 2024/11/24-08:40:29.453566 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297871.sst): old prealloc_size = 147639500, real_size = 32439856 2024/11/24-08:40:29.454180 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297871.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922181 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.54 avg-zval = 33.02 2024/11/24-08:40:29.455597 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297871.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:30.289834 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297871.sst): second pass time = 1.31's, 83.056'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.679'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2977.790'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 286.798'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30477.466'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 83.056'MB/sec zip pipeline throughput = 83.056'MB/sec entries = 922181 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.54 avg-zval = 33.02 usrkeys = 922181 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.54 avg-zval = 33.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922181 bytes = 3227648 } seq expand size = 6455267 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5137 value = 3.5593 dict = 1.00 all = 3.6746 } Zip/UnZip{ index = 0.0951 value = 0.2810 dict = 1.00 all = 0.2721 } ---------------------------- total value len = 2254.079459 GB avg = 0.199 KB (by entry num) total key len = 307.506085 GB avg = 0.027 KB total ukey len = 237.799815 GB avg = 0.015 KB total ukey num = 15.760915542 Billion total entry num = 11.302229490 Billion write speed all = 29.953732455 MB/sec (with seq num) write speed all = 28.896437424 MB/sec (without seq num) 2024/11/24-08:40:30.334686 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297871: 922181 keys, 36526184 bytes, temperature: kWarm 2024/11/24-08:40:30.334757 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408830334721, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297871, "file_size": 36526184, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4517407452, "table_properties": {"data_size": 30454880, "index_size": 1754240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25821068, "raw_average_key_size": 28, "raw_value_size": 108396798, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922181, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408827, "oldest_key_time": 0, "file_creation_time": 1732408827, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297871, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:30.371179 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297872.sst) /dev/shm/ToplingTemp/Topling-307465-3zb7Mg 2024/11/24-08:40:31.378277 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297872.sst): first pass time = 1.01's, 133.277'MB/sec 2024/11/24-08:40:32.677221 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297872.sst): old prealloc_size = 147639500, real_size = 31345024 2024/11/24-08:40:32.677747 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297872.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.93 2024/11/24-08:40:32.679673 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297872.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:33.495306 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297872.sst): second pass time = 1.30's, 83.563'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.159'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3517.689'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 277.447'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33813.575'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.563'MB/sec zip pipeline throughput = 83.563'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 31.93 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4436 value = 3.6954 dict = 1.00 all = 3.8015 } Zip/UnZip{ index = 0.0958 value = 0.2706 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 2254.194371 GB avg = 0.199 KB (by entry num) total key len = 307.531825 GB avg = 0.027 KB total ukey len = 237.818201 GB avg = 0.015 KB total ukey num = 15.761834842 Billion total entry num = 11.303148790 Billion write speed all = 29.954254395 MB/sec (with seq num) write speed all = 28.896912998 MB/sec (without seq num) 2024/11/24-08:40:33.537750 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297872: 919300 keys, 35306866 bytes, temperature: kWarm 2024/11/24-08:40:33.537823 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408833537786, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297872, "file_size": 35306866, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4361163805, "largest_seqno": 4473701318, "table_properties": {"data_size": 29354480, "index_size": 1760512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408830, "oldest_key_time": 0, "file_creation_time": 1732408830, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297872, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:33.575167 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297873.sst) /dev/shm/ToplingTemp/Topling-307465-pfZLae 2024/11/24-08:40:34.433322 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297873.sst): first pass time = 0.86's, 156.410'MB/sec 2024/11/24-08:40:35.751008 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297873.sst): old prealloc_size = 147639500, real_size = 30770176 2024/11/24-08:40:35.751587 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297873.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933250 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0287 GB avg-val = 115.82 avg-zval = 30.81 2024/11/24-08:40:35.752385 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297873.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:36.576810 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297873.sst): second pass time = 1.32's, 82.075'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.247'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3208.837'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 277.153'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31063.040'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.075'MB/sec zip pipeline throughput = 82.075'MB/sec entries = 933250 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.82 avg-zval = 30.81 usrkeys = 933250 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.82 avg-zval = 30.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933250 bytes = 3266384 } seq expand size = 6532750 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4441 value = 3.7596 dict = 1.00 all = 3.8472 } Zip/UnZip{ index = 0.0957 value = 0.2660 dict = 1.00 all = 0.2599 } ---------------------------- total value len = 2254.308991 GB avg = 0.199 KB (by entry num) total key len = 307.557956 GB avg = 0.027 KB total ukey len = 237.836866 GB avg = 0.015 KB total ukey num = 15.762768092 Billion total entry num = 11.304082040 Billion write speed all = 29.954820856 MB/sec (with seq num) write speed all = 28.897430260 MB/sec (without seq num) 2024/11/24-08:40:36.619960 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297873: 933250 keys, 34887304 bytes, temperature: kWarm 2024/11/24-08:40:36.620040 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408836620004, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297873, "file_size": 34887304, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4517407527, "table_properties": {"data_size": 28749552, "index_size": 1787136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26131000, "raw_average_key_size": 28, "raw_value_size": 108086760, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933250, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408833, "oldest_key_time": 0, "file_creation_time": 1732408833, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297873, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:36.656672 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297874.sst) /dev/shm/ToplingTemp/Topling-307465-oXyABg 2024/11/24-08:40:37.571209 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297874.sst): first pass time = 0.91's, 146.766'MB/sec 2024/11/24-08:40:38.883097 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297874.sst): old prealloc_size = 147639500, real_size = 31294272 2024/11/24-08:40:38.883667 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297874.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.88 2024/11/24-08:40:38.884946 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297874.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:39.695019 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297874.sst): second pass time = 1.31's, 82.759'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.425'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3228.943'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 275.192'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34468.835'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.759'MB/sec zip pipeline throughput = 82.759'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.88 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4443 value = 3.7018 dict = 1.00 all = 3.8069 } Zip/UnZip{ index = 0.0957 value = 0.2701 dict = 1.00 all = 0.2627 } ---------------------------- total value len = 2254.423903 GB avg = 0.199 KB (by entry num) total key len = 307.583697 GB avg = 0.027 KB total ukey len = 237.855252 GB avg = 0.015 KB total ukey num = 15.763687392 Billion total entry num = 11.305001340 Billion write speed all = 29.955373286 MB/sec (with seq num) write speed all = 28.897935251 MB/sec (without seq num) 2024/11/24-08:40:39.744177 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297874: 919300 keys, 35256114 bytes, temperature: kWarm 2024/11/24-08:40:39.744244 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408839744209, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297874, "file_size": 35256114, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4273975715, "largest_seqno": 4382547318, "table_properties": {"data_size": 29303856, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408836, "oldest_key_time": 0, "file_creation_time": 1732408836, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297874, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:39.782563 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297875.sst) /dev/shm/ToplingTemp/Topling-307465-QBtx8f 2024/11/24-08:40:40.786447 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297875.sst): first pass time = 1.00's, 133.703'MB/sec 2024/11/24-08:40:42.094903 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297875.sst): old prealloc_size = 147639500, real_size = 31433424 2024/11/24-08:40:42.095493 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297875.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.03 2024/11/24-08:40:42.096270 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297875.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:42.913552 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297875.sst): second pass time = 1.31's, 82.952'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.238'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3119.351'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 281.647'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36683.008'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.952'MB/sec zip pipeline throughput = 82.952'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.03 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4466 value = 3.6843 dict = 1.00 all = 3.7920 } Zip/UnZip{ index = 0.0957 value = 0.2714 dict = 1.00 all = 0.2637 } ---------------------------- total value len = 2254.538816 GB avg = 0.199 KB (by entry num) total key len = 307.609437 GB avg = 0.027 KB total ukey len = 237.873638 GB avg = 0.015 KB total ukey num = 15.764606692 Billion total entry num = 11.305920640 Billion write speed all = 29.955890530 MB/sec (with seq num) write speed all = 28.898406302 MB/sec (without seq num) 2024/11/24-08:40:42.958545 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297875: 919300 keys, 35395250 bytes, temperature: kWarm 2024/11/24-08:40:42.958618 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408842958582, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297875, "file_size": 35395250, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4382547319, "largest_seqno": 4495327897, "table_properties": {"data_size": 29443392, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408839, "oldest_key_time": 0, "file_creation_time": 1732408839, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297875, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:42.997049 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297876.sst) /dev/shm/ToplingTemp/Topling-307465-O0BJWh 2024/11/24-08:40:43.935867 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297876.sst): first pass time = 0.94's, 142.970'MB/sec 2024/11/24-08:40:45.260683 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297876.sst): old prealloc_size = 147639500, real_size = 31960368 2024/11/24-08:40:45.261299 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297876.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933383 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0300 GB avg-val = 115.80 avg-zval = 32.09 2024/11/24-08:40:45.262107 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297876.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:40:46.108468 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297876.sst): second pass time = 1.32's, 81.634'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.557'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2998.487'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 286.440'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35665.685'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.634'MB/sec zip pipeline throughput = 81.634'MB/sec entries = 933383 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.80 avg-zval = 32.09 usrkeys = 933383 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.80 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933383 bytes = 3266848 } seq expand size = 6533681 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5141 value = 3.6086 dict = 1.00 all = 3.7202 } Zip/UnZip{ index = 0.0951 value = 0.2771 dict = 1.00 all = 0.2688 } ---------------------------- total value len = 2254.653432 GB avg = 0.199 KB (by entry num) total key len = 307.635572 GB avg = 0.027 KB total ukey len = 237.892306 GB avg = 0.015 KB total ukey num = 15.765540075 Billion total entry num = 11.306854023 Billion write speed all = 29.956417162 MB/sec (with seq num) write speed all = 28.898885134 MB/sec (without seq num) 2024/11/24-08:40:46.153167 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297876: 933383 keys, 36077898 bytes, temperature: kWarm 2024/11/24-08:40:46.153243 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408846153204, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297876, "file_size": 36077898, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4517407090, "table_properties": {"data_size": 29951328, "index_size": 1775488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26134724, "raw_average_key_size": 28, "raw_value_size": 108083118, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933383, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408842, "oldest_key_time": 0, "file_creation_time": 1732408842, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297876, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:46.199526 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297877.sst) /dev/shm/ToplingTemp/Topling-307465-jHmwge 2024/11/24-08:40:46.338567 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297877.sst): first pass time = 0.14's, 242.466'MB/sec 2024/11/24-08:40:46.677549 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297877.sst): old prealloc_size = 147639500, real_size = 9446624 2024/11/24-08:40:46.677742 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297877.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 230843 prefix = 4 raw-key = 0.0037 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0272 GB zip-val = 0.0089 GB avg-val = 118.00 avg-zval = 38.75 2024/11/24-08:40:46.678363 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297877.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.034 GB 2024/11/24-08:40:46.889300 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00/297877.sst): second pass time = 0.34's, 80.544'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 49.083'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2635.903'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.323'MB/sec write SST data time = 0.03's, 313.607'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.280'MB, throughput = 8892.667'MB/sec zip my value time = 0.34's, unzip length = 0.027'GB zip my value throughput = 80.544'MB/sec zip pipeline throughput = 80.544'MB/sec entries = 230843 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 38.75 usrkeys = 230843 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 38.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 230843 bytes = 721392 } seq expand size = 1615901 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0046 GB value = 0.0272 GB dict = 0.28 MB all = 0.0337 GB } __ZipSize{ index = 0.0004 GB value = 0.0089 GB dict = 0.28 MB all = 0.0104 GB } UnZip/Zip{ index = 10.4156 value = 3.0451 dict = 1.00 all = 3.2431 } Zip/UnZip{ index = 0.0960 value = 0.3284 dict = 1.00 all = 0.3083 } ---------------------------- total value len = 2254.682288 GB avg = 0.199 KB (by entry num) total key len = 307.642036 GB avg = 0.027 KB total ukey len = 237.896922 GB avg = 0.015 KB total ukey num = 15.765770918 Billion total entry num = 11.307084866 Billion write speed all = 29.956556614 MB/sec (with seq num) write speed all = 28.899012649 MB/sec (without seq num) 2024/11/24-08:40:46.907356 139957517301312 [compaction_job.cc:1948] [default] [JOB 44088] Generated table #297877: 230843 keys, 10392335 bytes, temperature: kWarm 2024/11/24-08:40:46.907418 139957517301312 EVENT_LOG_v1 {"time_micros": 1732408846907387, "cf_name": "default", "job": 44088, "event": "table_file_creation", "file_number": 297877, "file_size": 10392335, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4293335505, "largest_seqno": 4320517749, "table_properties": {"data_size": 8945488, "index_size": 443264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6463604, "raw_average_key_size": 28, "raw_value_size": 27239474, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 230843, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732408846, "oldest_key_time": 0, "file_creation_time": 1732408846, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297877, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:40:46.921655 139957517301312 [compaction_job.cc:703] job-44088: subcompact[0], size: 1.036191 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-44088/att-00 [297848,297849,297850,297851,297852,297853,297854,297855,297856,297857,297858,297859,297860,297861,297862,297863,297864,297865,297866,297867,297868,297869,297870,297871,297872,297873,297874,297875,297876,297877] 2024/11/24-08:40:47.095761 139957517301312 (Original Log Time 2024/11/24-08:39:13.718287) EVENT_LOG_v1 {"time_micros": 1732408753718246, "job": 44088, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [297808], "files_L3": [297754, 297755, 297756, 297757, 297758, 297759, 297760, 297761, 297762, 297763, 297764, 297765, 297766, 297767, 297768, 297769, 297770, 297771, 297772, 297773, 297774, 297775, 297776, 297777, 297778, 297779, 297780, 297731, 297732, 297839], "score": 0.805725, "input_data_size": 1059732015, "oldest_snapshot_seqno": 4517543758} 2024/11/24-08:40:47.095765 139957517301312 (Original Log Time 2024/11/24-08:40:47.094058) [compaction_job.cc:2022] [default] [JOB 44088] Compacted 1@2 + 30@3 files to L3 => 1036190509 bytes 2024/11/24-08:40:47.095766 139957517301312 (Original Log Time 2024/11/24-08:40:47.095680) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 30 0 0 0] max score 0.04, MB/sec: 11.4 rd, 11.1 wr, level 3, files in(1, 30) out(30 +0 blob) MB in(27.1, 983.6 +0.0 blob) out(988.2 +0.0 blob), read-write-amplify(73.8) write-amplify(36.5) OK, records in: 27214008, records dropped: 104253 output_compression: Snappy 2024/11/24-08:40:47.095769 139957517301312 (Original Log Time 2024/11/24-08:40:47.095734) EVENT_LOG_v1 {"time_micros": 1732408847095710, "cf": "default", "job": 44088, "event": "compaction_finished", "compaction_time_micros": 93203376, "compaction_time_cpu_micros": 40583615, "output_level": 3, "num_output_files": 30, "total_output_size": 1036190509, "num_input_records": 27214008, "num_output_records": 27109755, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 723386042, "file_range_sync_nanos": 0, "file_fsync_nanos": 30848, "file_prepare_write_nanos": 74349, "lsm_state": [0, 0, 0, 30, 0, 0, 0]} 2024/11/24-08:40:47.232455 139957517301312 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732408753,"executesMs":93529,"compactionJobId":44088,"attempt":0,"compactionInputRawBytes":3940823404,"compactionInputZipBytes":1059732015,"compactionOutputRawBytes":0,"compactionOutputZipBytes":1036190509,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-2ab4d21dd1f0483cfa5b100d210982c9-a8c005626acf497d-00"}}, response = 2024/11/24-08:40:47.232529 139957517301312 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-44088/att-00: olev 3, work 93.529 s, result 7.731 ms, install 1.726 ms, input{raw 3.670 GiB zip 1010.639 MiB}