2024/11/23-18:02:16.231742 139957500515904 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00: fork to child time = 0.014515 sec 2024/11/23-18:02:16.232439 139957500515904 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00 2024/11/23-18:02:16.239304 139957500515904 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00 2024/11/23-18:02:16.240148 139957500515904 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-18:02:16.240353 139957500515904 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-18:02:16.291929 139957500515904 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32608/att-00/MANIFEST-206347 2024/11/23-18:02:16.316632 139957500515904 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32608/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 273762, last_sequence is 4168166213, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 273598 2024/11/23-18:02:16.316640 139957500515904 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-18:02:16.316763 139957500515904 [version_set.cc:5721] Creating manifest 273762 2024/11/23-18:02:16.353371 139957500515904 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-18:02:16.373507 139957500515904 [compaction_job.cc:2437] [default] [JOB 32608] Compacting 1@2 + 21@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-18:02:16.373522 139957500515904 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24415 Base level 2, inputs: [273734(27MB)], [273754(37MB) 273755(4142KB) 273683(38MB) 273685(34MB) 273686(35MB) 273687(38MB) 273688(35MB) 273689(36MB) 273690(34MB) 273691(35MB) 273692(37MB) 273693(35MB) 273694(36MB) 273695(35MB) 273696(35MB) 273697(36MB) 273698(37MB) 273699(37MB) 273700(36MB) 273701(9163KB) 273756(34MB)] 2024/11/23-18:02:16.380113 139957500515904 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:02:16.380144 139957500515904 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:02:16.380158 139957500515904 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:02:16.380163 139957500515904 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:02:16.380173 139957500515904 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:02:16.940271 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273763.sst) /dev/shm/ToplingTemp/Topling-142732-NGAj6U 2024/11/23-18:02:17.808882 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273763.sst): first pass time = 0.87's, 154.528'MB/sec 2024/11/23-18:02:21.127815 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273763.sst): old prealloc_size = 147639500, real_size = 35473424 2024/11/23-18:02:21.128400 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273763.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930481 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.25 avg-zval = 35.96 2024/11/23-18:02:21.129233 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273763.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:02:22.082668 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273763.sst): second pass time = 3.32's, 32.599'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 41.458'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3150.445'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.23's, 169.804'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35726.691'MB/sec zip my value time = 3.32's, unzip length = 0.108'GB zip my value throughput = 32.599'MB/sec zip pipeline throughput = 111.287'MB/sec entries = 930481 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.25 avg-zval = 35.96 usrkeys = 930481 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.25 avg-zval = 35.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930481 bytes = 3256704 } seq expand size = 6513367 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4460 value = 3.2327 dict = 1.00 all = 3.3906 } Zip/UnZip{ index = 0.0957 value = 0.3093 dict = 1.00 all = 0.2949 } ---------------------------- total value len = 8.963297 GB avg = 0.168 KB (by entry num) total key len = 1.466875 GB avg = 0.028 KB total ukey len = 1.016488 GB avg = 0.015 KB total ukey num = 0.066480916 Billion total entry num = 0.053288645 Billion write speed all = 0.317703745 MB/sec (with seq num) write speed all = 0.304718339 MB/sec (without seq num) 2024/11/23-18:02:22.139221 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273763: 930481 keys, 39584998 bytes, temperature: kWarm 2024/11/23-18:02:22.139350 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356142139275, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273763, "file_size": 39584998, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4003482220, "largest_seqno": 4167925630, "table_properties": {"data_size": 33459136, "index_size": 1781504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26053468, "raw_average_key_size": 28, "raw_value_size": 108164298, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930481, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356136, "oldest_key_time": 0, "file_creation_time": 1732356136, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273763, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:02:22.225825 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273764.sst) /dev/shm/ToplingTemp/Topling-142732-Gg0kqU 2024/11/23-18:02:23.286582 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273764.sst): first pass time = 1.06's, 126.537'MB/sec 2024/11/23-18:02:26.072029 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273764.sst): old prealloc_size = 147639500, real_size = 36518480 2024/11/23-18:02:26.072636 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273764.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933945 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0345 GB avg-val = 115.71 avg-zval = 36.95 2024/11/23-18:02:26.073366 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273764.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:02:27.006225 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273764.sst): second pass time = 2.78's, 38.808'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 42.953'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3072.464'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.335'MB/sec write SST data time = 0.24's, 170.136'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35383.131'MB/sec zip my value time = 2.78's, unzip length = 0.108'GB zip my value throughput = 38.808'MB/sec zip pipeline throughput = 104.083'MB/sec entries = 933945 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.71 avg-zval = 36.95 usrkeys = 933945 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.10 avg-zval = 36.95 TagRS{ kinds = RS_Key0_TagN bytes = 146040 } TagArray{ size = 719011 bytes = 2606432 } seq expand size = 5033077 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.5136 value = 3.1316 dict = 1.00 all = 3.3450 } Zip/UnZip{ index = 0.0951 value = 0.3193 dict = 1.00 all = 0.2990 } ---------------------------- total value len = 9.076398 GB avg = 0.167 KB (by entry num) total key len = 1.493025 GB avg = 0.028 KB total ukey len = 1.035167 GB avg = 0.015 KB total ukey num = 0.067414861 Billion total entry num = 0.054222590 Billion write speed all = 0.321897059 MB/sec (with seq num) write speed all = 0.308686050 MB/sec (without seq num) 2024/11/23-18:02:27.065793 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273764: 933945 keys, 40125244 bytes, temperature: kWarm 2024/11/23-18:02:27.065886 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356147065838, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273764, "file_size": 40125244, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34508160, "index_size": 1776640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26150460, "raw_average_key_size": 28, "raw_value_size": 108067298, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933945, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356142, "oldest_key_time": 0, "file_creation_time": 1732356142, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273764, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:02:27.134839 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273765.sst) /dev/shm/ToplingTemp/Topling-142732-C8OHsX 2024/11/23-18:02:28.446036 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273765.sst): first pass time = 1.31's, 102.367'MB/sec 2024/11/23-18:02:30.384966 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273765.sst): old prealloc_size = 147639500, real_size = 33430640 2024/11/23-18:02:30.385552 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273765.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929952 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.33 avg-zval = 33.78 2024/11/23-18:02:30.386354 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273765.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:02:31.405522 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273765.sst): second pass time = 1.94's, 55.814'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 36.820'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3220.041'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.24's, 159.356'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 38168.288'MB/sec zip my value time = 1.94's, unzip length = 0.108'GB zip my value throughput = 55.814'MB/sec zip pipeline throughput = 61.692'MB/sec entries = 929952 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.33 avg-zval = 33.78 usrkeys = 929952 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.64 avg-zval = 33.78 TagRS{ kinds = RS_Key0_TagN bytes = 145400 } TagArray{ size = 838094 bytes = 3142864 } seq expand size = 5866658 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4431 value = 3.4433 dict = 1.00 all = 3.5720 } Zip/UnZip{ index = 0.0958 value = 0.2904 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 9.190444 GB avg = 0.167 KB (by entry num) total key len = 1.519064 GB avg = 0.028 KB total ukey len = 1.053766 GB avg = 0.015 KB total ukey num = 0.068344813 Billion total entry num = 0.055152542 Billion write speed all = 0.326119708 MB/sec (with seq num) write speed all = 0.312683922 MB/sec (without seq num) 2024/11/23-18:02:31.457188 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273765: 929952 keys, 37575148 bytes, temperature: kWarm 2024/11/23-18:02:31.457292 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356151457233, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273765, "file_size": 37575148, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775918, "table_properties": {"data_size": 31416992, "index_size": 1780992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26038656, "raw_average_key_size": 28, "raw_value_size": 108179188, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929952, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356147, "oldest_key_time": 0, "file_creation_time": 1732356147, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273765, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:02:31.530551 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273766.sst) /dev/shm/ToplingTemp/Topling-142732-1uc9lU 2024/11/23-18:02:32.593072 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273766.sst): first pass time = 1.06's, 126.327'MB/sec 2024/11/23-18:02:34.160261 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273766.sst): old prealloc_size = 147639500, real_size = 33214432 2024/11/23-18:02:34.160775 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273766.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926255 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.90 avg-zval = 33.71 2024/11/23-18:02:34.162366 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273766.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:02:35.227193 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273766.sst): second pass time = 1.57's, 69.148'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 35.188'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3579.922'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.247'MB/sec write SST data time = 0.13's, 294.111'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 40584.451'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 69.148'MB/sec zip pipeline throughput = 69.148'MB/sec entries = 926255 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.90 avg-zval = 33.71 usrkeys = 926255 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.90 avg-zval = 33.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926255 bytes = 3473472 } seq expand size = 6483785 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5138 value = 3.4683 dict = 1.00 all = 3.5751 } Zip/UnZip{ index = 0.0951 value = 0.2883 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 9.305210 GB avg = 0.166 KB (by entry num) total key len = 1.544999 GB avg = 0.028 KB total ukey len = 1.072291 GB avg = 0.015 KB total ukey num = 0.069271068 Billion total entry num = 0.056078797 Billion write speed all = 0.330365824 MB/sec (with seq num) write speed all = 0.316705982 MB/sec (without seq num) 2024/11/23-18:02:35.283733 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273766: 926255 keys, 37542234 bytes, temperature: kWarm 2024/11/23-18:02:35.283804 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356155283768, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273766, "file_size": 37542234, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012774578, "table_properties": {"data_size": 31220688, "index_size": 1761984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935140, "raw_average_key_size": 28, "raw_value_size": 108282638, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926255, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356151, "oldest_key_time": 0, "file_creation_time": 1732356151, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273766, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:02:35.323467 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273767.sst) /dev/shm/ToplingTemp/Topling-142732-qIGXbX 2024/11/23-18:02:36.235568 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273767.sst): first pass time = 0.91's, 147.158'MB/sec 2024/11/23-18:02:37.893930 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273767.sst): old prealloc_size = 147639500, real_size = 35190656 2024/11/23-18:02:37.894640 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273767.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929411 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0332 GB avg-val = 116.41 avg-zval = 35.70 2024/11/23-18:02:37.895548 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273767.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:02:38.937173 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273767.sst): second pass time = 1.66's, 65.281'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 37.779'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2645.641'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.261'MB/sec write SST data time = 0.15's, 256.092'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36803.356'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.281'MB/sec zip pipeline throughput = 65.281'MB/sec entries = 929411 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.41 avg-zval = 35.70 usrkeys = 929411 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.41 avg-zval = 35.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929411 bytes = 3369136 } seq expand size = 6505877 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4445 value = 3.2610 dict = 1.00 all = 3.4055 } Zip/UnZip{ index = 0.0957 value = 0.3067 dict = 1.00 all = 0.2936 } ---------------------------- total value len = 9.419910 GB avg = 0.165 KB (by entry num) total key len = 1.571022 GB avg = 0.028 KB total ukey len = 1.090879 GB avg = 0.016 KB total ukey num = 0.070200479 Billion total entry num = 0.057008208 Billion write speed all = 0.334612763 MB/sec (with seq num) write speed all = 0.320728100 MB/sec (without seq num) 2024/11/23-18:02:38.985365 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273767: 929411 keys, 39412476 bytes, temperature: kWarm 2024/11/23-18:02:38.985432 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356158985397, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273767, "file_size": 39412476, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33178416, "index_size": 1779712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26023508, "raw_average_key_size": 28, "raw_value_size": 108194282, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929411, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356155, "oldest_key_time": 0, "file_creation_time": 1732356155, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273767, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:02:39.026038 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273768.sst) /dev/shm/ToplingTemp/Topling-142732-5yzgJT 2024/11/23-18:02:39.991107 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273768.sst): first pass time = 0.97's, 139.082'MB/sec 2024/11/23-18:02:41.499345 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273768.sst): old prealloc_size = 147639500, real_size = 32674272 2024/11/23-18:02:41.499882 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273768.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928540 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.55 avg-zval = 33.02 2024/11/23-18:02:41.500738 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273768.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:02:42.355832 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273768.sst): second pass time = 1.51's, 71.791'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.212'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3441.421'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 286.233'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28056.029'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.791'MB/sec zip pipeline throughput = 71.791'MB/sec entries = 928540 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.55 avg-zval = 33.02 usrkeys = 928540 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.55 avg-zval = 33.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928540 bytes = 3482032 } seq expand size = 6499780 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4464 value = 3.5292 dict = 1.00 all = 3.6266 } Zip/UnZip{ index = 0.0957 value = 0.2834 dict = 1.00 all = 0.2757 } ---------------------------- total value len = 9.534629 GB avg = 0.165 KB (by entry num) total key len = 1.597021 GB avg = 0.028 KB total ukey len = 1.109450 GB avg = 0.016 KB total ukey num = 0.071129019 Billion total entry num = 0.057936748 Billion write speed all = 0.338861560 MB/sec (with seq num) write speed all = 0.324752215 MB/sec (without seq num) 2024/11/23-18:02:42.402600 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273768: 928540 keys, 37009532 bytes, temperature: kWarm 2024/11/23-18:02:42.402667 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356162402633, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273768, "file_size": 37009532, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30664208, "index_size": 1777728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25999120, "raw_average_key_size": 28, "raw_value_size": 108218624, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928540, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356159, "oldest_key_time": 0, "file_creation_time": 1732356159, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273768, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:02:42.441777 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273769.sst) /dev/shm/ToplingTemp/Topling-142732-Rr66CX 2024/11/23-18:02:43.364305 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273769.sst): first pass time = 0.92's, 145.496'MB/sec 2024/11/23-18:02:44.720179 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273769.sst): old prealloc_size = 147639500, real_size = 35180112 2024/11/23-18:02:44.720707 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273769.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924948 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 117.11 avg-zval = 35.87 2024/11/23-18:02:44.721428 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273769.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:02:45.537085 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273769.sst): second pass time = 1.35's, 79.955'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 47.702'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3510.030'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 301.068'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33431.018'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.955'MB/sec zip pipeline throughput = 79.955'MB/sec entries = 924948 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.11 avg-zval = 35.87 usrkeys = 924948 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.72 avg-zval = 35.87 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 741951 bytes = 2782336 } seq expand size = 5193657 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4443 value = 3.2648 dict = 1.00 all = 3.4444 } Zip/UnZip{ index = 0.0957 value = 0.3063 dict = 1.00 all = 0.2903 } ---------------------------- total value len = 9.648142 GB avg = 0.164 KB (by entry num) total key len = 1.622920 GB avg = 0.028 KB total ukey len = 1.127949 GB avg = 0.016 KB total ukey num = 0.072053967 Billion total entry num = 0.058861696 Billion write speed all = 0.343072197 MB/sec (with seq num) write speed all = 0.328738987 MB/sec (without seq num) 2024/11/23-18:02:45.585954 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273769: 924948 keys, 38966490 bytes, temperature: kWarm 2024/11/23-18:02:45.586040 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356165586004, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273769, "file_size": 38966490, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 33177472, "index_size": 1771200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25898544, "raw_average_key_size": 28, "raw_value_size": 108319220, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924948, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356162, "oldest_key_time": 0, "file_creation_time": 1732356162, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273769, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:02:45.624607 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273770.sst) /dev/shm/ToplingTemp/Topling-142732-aWst4S 2024/11/23-18:02:46.511555 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273770.sst): first pass time = 0.89's, 151.333'MB/sec 2024/11/23-18:02:47.821171 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273770.sst): old prealloc_size = 147639500, real_size = 32091600 2024/11/23-18:02:47.821722 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273770.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0301 GB avg-val = 115.79 avg-zval = 32.23 2024/11/23-18:02:47.822545 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273770.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:02:48.683026 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273770.sst): second pass time = 1.31's, 82.589'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.151'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3376.056'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.317'MB/sec write SST data time = 0.13's, 287.787'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30199.283'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.589'MB/sec zip pipeline throughput = 82.589'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.79 avg-zval = 32.23 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 32.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5153 value = 3.5928 dict = 1.00 all = 3.6948 } Zip/UnZip{ index = 0.0951 value = 0.2783 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 9.762757 GB avg = 0.163 KB (by entry num) total key len = 1.649057 GB avg = 0.028 KB total ukey len = 1.146618 GB avg = 0.016 KB total ukey num = 0.072987424 Billion total entry num = 0.059795153 Billion write speed all = 0.347323195 MB/sec (with seq num) write speed all = 0.332764076 MB/sec (without seq num) 2024/11/23-18:02:48.728409 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273770: 933457 keys, 36325942 bytes, temperature: kWarm 2024/11/23-18:02:48.728481 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356168728443, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273770, "file_size": 36325942, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 30082624, "index_size": 1775424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356165, "oldest_key_time": 0, "file_creation_time": 1732356165, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273770, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:02:48.764588 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273771.sst) /dev/shm/ToplingTemp/Topling-142732-dZNdaX 2024/11/23-18:02:49.594880 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273771.sst): first pass time = 0.83's, 161.660'MB/sec 2024/11/23-18:02:50.913223 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273771.sst): old prealloc_size = 147639500, real_size = 33012416 2024/11/23-18:02:50.913764 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273771.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933113 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0310 GB avg-val = 115.84 avg-zval = 33.21 2024/11/23-18:02:50.914618 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273771.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:02:51.763776 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273771.sst): second pass time = 1.32's, 82.062'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.791'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3521.906'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 291.066'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 35420.053'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.062'MB/sec zip pipeline throughput = 82.062'MB/sec entries = 933113 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.84 avg-zval = 33.21 usrkeys = 933113 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.84 avg-zval = 33.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933113 bytes = 3382544 } seq expand size = 6531791 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4455 value = 3.4877 dict = 1.00 all = 3.6036 } Zip/UnZip{ index = 0.0957 value = 0.2867 dict = 1.00 all = 0.2775 } ---------------------------- total value len = 9.877379 GB avg = 0.163 KB (by entry num) total key len = 1.675184 GB avg = 0.028 KB total ukey len = 1.165280 GB avg = 0.016 KB total ukey num = 0.073920537 Billion total entry num = 0.060728266 Billion write speed all = 0.351574000 MB/sec (with seq num) write speed all = 0.336789071 MB/sec (without seq num) 2024/11/23-18:02:51.812641 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273771: 933113 keys, 37245174 bytes, temperature: kWarm 2024/11/23-18:02:51.812716 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356171812678, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273771, "file_size": 37245174, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30992304, "index_size": 1786624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26127164, "raw_average_key_size": 28, "raw_value_size": 108090618, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933113, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356168, "oldest_key_time": 0, "file_creation_time": 1732356168, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273771, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:02:51.850774 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273772.sst) /dev/shm/ToplingTemp/Topling-142732-X1H9CW 2024/11/23-18:02:52.756023 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273772.sst): first pass time = 0.91's, 148.274'MB/sec 2024/11/23-18:02:54.079967 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273772.sst): old prealloc_size = 147639500, real_size = 34931728 2024/11/23-18:02:54.080579 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273772.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0329 GB avg-val = 117.01 avg-zval = 35.56 2024/11/23-18:02:54.081960 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273772.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:02:54.923287 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273772.sst): second pass time = 1.32's, 81.853'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.553'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3183.476'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 311.121'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30601.660'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.853'MB/sec zip pipeline throughput = 81.853'MB/sec entries = 925568 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.01 avg-zval = 35.56 usrkeys = 925568 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.01 avg-zval = 35.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925568 bytes = 3470896 } seq expand size = 6478976 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.3752 value = 3.2903 dict = 1.00 all = 3.4189 } Zip/UnZip{ index = 0.0964 value = 0.3039 dict = 1.00 all = 0.2925 } ---------------------------- total value len = 9.992160 GB avg = 0.162 KB (by entry num) total key len = 1.701100 GB avg = 0.028 KB total ukey len = 1.183791 GB avg = 0.016 KB total ukey num = 0.074846105 Billion total entry num = 0.061653834 Billion write speed all = 0.355821547 MB/sec (with seq num) write speed all = 0.340812722 MB/sec (without seq num) 2024/11/23-18:02:54.970839 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273772: 925568 keys, 39257554 bytes, temperature: kWarm 2024/11/23-18:02:54.970911 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356174970874, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273772, "file_size": 39257554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 32915968, "index_size": 1784192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915904, "raw_average_key_size": 28, "raw_value_size": 108301824, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356171, "oldest_key_time": 0, "file_creation_time": 1732356171, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273772, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:02:55.034081 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273773.sst) /dev/shm/ToplingTemp/Topling-142732-wsWWAU 2024/11/23-18:02:56.052674 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273773.sst): first pass time = 1.02's, 131.773'MB/sec 2024/11/23-18:02:57.396623 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273773.sst): old prealloc_size = 147639500, real_size = 32637984 2024/11/23-18:02:57.397219 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273773.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.13 avg-zval = 32.89 2024/11/23-18:02:57.398051 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273773.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:02:58.251264 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273773.sst): second pass time = 1.34's, 80.513'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.137'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3063.188'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 287.865'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34057.468'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.513'MB/sec zip pipeline throughput = 80.513'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 32.89 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 32.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5116 value = 3.5303 dict = 1.00 all = 3.6408 } Zip/UnZip{ index = 0.0951 value = 0.2833 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 10.106822 GB avg = 0.161 KB (by entry num) total key len = 1.727175 GB avg = 0.028 KB total ukey len = 1.202416 GB avg = 0.016 KB total ukey num = 0.075777347 Billion total entry num = 0.062585076 Billion write speed all = 0.360067641 MB/sec (with seq num) write speed all = 0.344833659 MB/sec (without seq num) 2024/11/23-18:02:58.299081 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273773: 931242 keys, 36865368 bytes, temperature: kWarm 2024/11/23-18:02:58.299156 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356178299119, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273773, "file_size": 36865368, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 30633168, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356175, "oldest_key_time": 0, "file_creation_time": 1732356175, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273773, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:02:58.343464 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273774.sst) /dev/shm/ToplingTemp/Topling-142732-6iwf2U 2024/11/23-18:02:59.312708 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273774.sst): first pass time = 0.97's, 138.483'MB/sec 2024/11/23-18:03:00.649744 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273774.sst): old prealloc_size = 147639500, real_size = 34677808 2024/11/23-18:03:00.650351 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273774.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0327 GB avg-val = 117.46 avg-zval = 35.43 2024/11/23-18:03:00.651189 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273774.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:03:01.494093 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273774.sst): second pass time = 1.34's, 81.111'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.186'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3004.042'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 305.827'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33907.747'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.111'MB/sec zip pipeline throughput = 81.111'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 35.43 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 35.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5131 value = 3.3153 dict = 1.00 all = 3.4418 } Zip/UnZip{ index = 0.0951 value = 0.3016 dict = 1.00 all = 0.2905 } ---------------------------- total value len = 10.221663 GB avg = 0.161 KB (by entry num) total key len = 1.753010 GB avg = 0.028 KB total ukey len = 1.220870 GB avg = 0.016 KB total ukey num = 0.076700044 Billion total entry num = 0.063507773 Billion write speed all = 0.364312001 MB/sec (with seq num) write speed all = 0.348854948 MB/sec (without seq num) 2024/11/23-18:03:01.543157 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273774: 922697 keys, 38995914 bytes, temperature: kWarm 2024/11/23-18:03:01.543231 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356181543195, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273774, "file_size": 38995914, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 32691616, "index_size": 1755328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356178, "oldest_key_time": 0, "file_creation_time": 1732356178, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273774, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:03:01.582144 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273775.sst) /dev/shm/ToplingTemp/Topling-142732-7CuEnU 2024/11/23-18:03:02.618594 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273775.sst): first pass time = 1.04's, 129.505'MB/sec 2024/11/23-18:03:03.965333 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273775.sst): old prealloc_size = 147639500, real_size = 32965040 2024/11/23-18:03:03.965859 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273775.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 116.15 avg-zval = 33.23 2024/11/23-18:03:03.966761 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273775.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:03:04.827630 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273775.sst): second pass time = 1.35's, 80.358'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.267'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3542.341'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 292.004'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 23747.219'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.358'MB/sec zip pipeline throughput = 80.358'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.15 avg-zval = 33.23 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.15 avg-zval = 33.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.3756 value = 3.4957 dict = 1.00 all = 3.5976 } Zip/UnZip{ index = 0.0964 value = 0.2861 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 10.336328 GB avg = 0.160 KB (by entry num) total key len = 1.779080 GB avg = 0.028 KB total ukey len = 1.239492 GB avg = 0.016 KB total ukey num = 0.077631127 Billion total entry num = 0.064438856 Billion write speed all = 0.368556293 MB/sec (with seq num) write speed all = 0.352874216 MB/sec (without seq num) 2024/11/23-18:03:04.873202 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273775: 931083 keys, 37307684 bytes, temperature: kWarm 2024/11/23-18:03:04.873280 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356184873235, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273775, "file_size": 37307684, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30937312, "index_size": 1794752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356181, "oldest_key_time": 0, "file_creation_time": 1732356181, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273775, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:03:04.913229 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273776.sst) /dev/shm/ToplingTemp/Topling-142732-rQIVsU 2024/11/23-18:03:05.918045 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273776.sst): first pass time = 1.00's, 133.580'MB/sec 2024/11/23-18:03:07.315924 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273776.sst): old prealloc_size = 147639500, real_size = 34822976 2024/11/23-18:03:07.316518 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273776.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928635 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0328 GB avg-val = 116.53 avg-zval = 35.33 2024/11/23-18:03:07.317020 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273776.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:03:08.104409 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273776.sst): second pass time = 1.40's, 77.459'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 47.617'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3099.789'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.352'MB/sec write SST data time = 0.13's, 293.881'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33596.317'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.459'MB/sec zip pipeline throughput = 77.459'MB/sec entries = 928635 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.53 avg-zval = 35.33 usrkeys = 928635 avg-key = 20.00 avg-zkey = 1.92 avg-val = 120.23 avg-zval = 35.33 TagRS{ kinds = RS_Key0_TagN bytes = 145200 } TagArray{ size = 490796 bytes = 1717808 } seq expand size = 3435572 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4425 value = 3.2981 dict = 1.00 all = 3.5749 } Zip/UnZip{ index = 0.0958 value = 0.3032 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 10.447979 GB avg = 0.160 KB (by entry num) total key len = 1.805082 GB avg = 0.028 KB total ukey len = 1.258065 GB avg = 0.016 KB total ukey num = 0.078559762 Billion total entry num = 0.065367491 Billion write speed all = 0.372706619 MB/sec (with seq num) write speed all = 0.356800132 MB/sec (without seq num) 2024/11/23-18:03:08.151578 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273776: 928635 keys, 37544359 bytes, temperature: kWarm 2024/11/23-18:03:08.151653 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356188151616, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273776, "file_size": 37544359, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 32812080, "index_size": 1778560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26001780, "raw_average_key_size": 28, "raw_value_size": 108215982, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928635, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356184, "oldest_key_time": 0, "file_creation_time": 1732356184, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273776, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:03:08.189440 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273777.sst) /dev/shm/ToplingTemp/Topling-142732-GFaEgX 2024/11/23-18:03:09.174943 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273777.sst): first pass time = 0.99's, 136.197'MB/sec 2024/11/23-18:03:10.685068 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273777.sst): old prealloc_size = 147639500, real_size = 36872144 2024/11/23-18:03:10.685669 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273777.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926735 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.83 avg-zval = 37.62 2024/11/23-18:03:10.685934 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273777.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:03:11.430098 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273777.sst): second pass time = 1.51's, 71.736'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.75's, 51.486'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3048.955'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.389'MB/sec write SST data time = 0.15's, 257.199'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29988.675'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.736'MB/sec zip pipeline throughput = 71.736'MB/sec entries = 926735 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.83 avg-zval = 37.62 usrkeys = 926735 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.99 avg-zval = 37.62 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4464 value = 3.1053 dict = 1.00 all = 3.4991 } Zip/UnZip{ index = 0.0957 value = 0.3220 dict = 1.00 all = 0.2858 } ---------------------------- total value len = 10.557330 GB avg = 0.159 KB (by entry num) total key len = 1.831031 GB avg = 0.028 KB total ukey len = 1.276599 GB avg = 0.016 KB total ukey num = 0.079486497 Billion total entry num = 0.066294226 Billion write speed all = 0.376783941 MB/sec (with seq num) write speed all = 0.360653574 MB/sec (without seq num) 2024/11/23-18:03:11.443005 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273777: 926735 keys, 38357733 bytes, temperature: kWarm 2024/11/23-18:03:11.443077 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356191443043, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273777, "file_size": 38357733, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34865984, "index_size": 1774272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25948580, "raw_average_key_size": 28, "raw_value_size": 108269286, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926735, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356188, "oldest_key_time": 0, "file_creation_time": 1732356188, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273777, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:03:11.478572 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273778.sst) /dev/shm/ToplingTemp/Topling-142732-J95YBT 2024/11/23-18:03:12.547315 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273778.sst): first pass time = 1.07's, 125.589'MB/sec 2024/11/23-18:03:13.891969 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273778.sst): old prealloc_size = 147639500, real_size = 35243424 2024/11/23-18:03:13.892632 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273778.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928889 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0333 GB avg-val = 116.49 avg-zval = 35.80 2024/11/23-18:03:13.893388 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273778.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:03:14.713003 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273778.sst): second pass time = 1.34's, 80.521'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 47.718'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2913.756'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 304.414'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32968.016'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.521'MB/sec zip pipeline throughput = 80.521'MB/sec entries = 928889 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.49 avg-zval = 35.80 usrkeys = 928889 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.80 avg-zval = 35.80 TagRS{ kinds = RS_Key0_TagN bytes = 145240 } TagArray{ size = 837472 bytes = 2931168 } seq expand size = 5862304 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5840 value = 3.2538 dict = 1.00 all = 3.4260 } Zip/UnZip{ index = 0.0945 value = 0.3073 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 10.671401 GB avg = 0.159 KB (by entry num) total key len = 1.857040 GB avg = 0.028 KB total ukey len = 1.295177 GB avg = 0.016 KB total ukey num = 0.080415386 Billion total entry num = 0.067223115 Billion write speed all = 0.381006343 MB/sec (with seq num) write speed all = 0.364651596 MB/sec (without seq num) 2024/11/23-18:03:14.759892 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273778: 928889 keys, 39176376 bytes, temperature: kWarm 2024/11/23-18:03:14.759964 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356194759928, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273778, "file_size": 39176376, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4167925733, "table_properties": {"data_size": 33255760, "index_size": 1755264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26008892, "raw_average_key_size": 28, "raw_value_size": 108208866, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928889, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356191, "oldest_key_time": 0, "file_creation_time": 1732356191, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273778, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:03:14.801615 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273779.sst) /dev/shm/ToplingTemp/Topling-142732-TeumkX 2024/11/23-18:03:15.669255 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273779.sst): first pass time = 0.87's, 154.700'MB/sec 2024/11/23-18:03:17.011882 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273779.sst): old prealloc_size = 147639500, real_size = 34865552 2024/11/23-18:03:17.012499 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273779.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926484 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0329 GB avg-val = 116.87 avg-zval = 35.48 2024/11/23-18:03:17.013235 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273779.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:03:17.847725 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273779.sst): second pass time = 1.34's, 80.738'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.339'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2971.898'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 309.134'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33306.246'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.738'MB/sec zip pipeline throughput = 80.738'MB/sec entries = 926484 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 35.48 usrkeys = 926484 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 35.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926484 bytes = 3011088 } seq expand size = 6485388 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5133 value = 3.2940 dict = 1.00 all = 3.4654 } Zip/UnZip{ index = 0.0951 value = 0.3036 dict = 1.00 all = 0.2886 } ---------------------------- total value len = 10.786162 GB avg = 0.158 KB (by entry num) total key len = 1.882981 GB avg = 0.028 KB total ukey len = 1.313707 GB avg = 0.016 KB total ukey num = 0.081341870 Billion total entry num = 0.068149599 Billion write speed all = 0.385248584 MB/sec (with seq num) write speed all = 0.368670014 MB/sec (without seq num) 2024/11/23-18:03:17.894791 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273779: 926484 keys, 38731220 bytes, temperature: kWarm 2024/11/23-18:03:17.894864 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356197894828, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273779, "file_size": 38731220, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4101998674, "largest_seqno": 4167925883, "table_properties": {"data_size": 32871232, "index_size": 1762496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941552, "raw_average_key_size": 28, "raw_value_size": 108276208, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926484, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356194, "oldest_key_time": 0, "file_creation_time": 1732356194, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273779, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:03:17.931599 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273780.sst) /dev/shm/ToplingTemp/Topling-142732-g3QyGW 2024/11/23-18:03:18.784493 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273780.sst): first pass time = 0.85's, 157.374'MB/sec 2024/11/23-18:03:20.121853 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273780.sst): old prealloc_size = 147639500, real_size = 34470768 2024/11/23-18:03:20.122458 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273780.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925131 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 117.08 avg-zval = 35.11 2024/11/23-18:03:20.123210 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273780.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:03:20.964538 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273780.sst): second pass time = 1.34's, 81.065'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.628'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3024.091'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 307.064'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34328.263'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.065'MB/sec zip pipeline throughput = 81.065'MB/sec entries = 925131 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 35.11 usrkeys = 925131 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 35.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925131 bytes = 3122336 } seq expand size = 6475917 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5129 value = 3.3349 dict = 1.00 all = 3.4908 } Zip/UnZip{ index = 0.0951 value = 0.2999 dict = 1.00 all = 0.2865 } ---------------------------- total value len = 10.900952 GB avg = 0.158 KB (by entry num) total key len = 1.908885 GB avg = 0.028 KB total ukey len = 1.332209 GB avg = 0.016 KB total ukey num = 0.082267001 Billion total entry num = 0.069074730 Billion write speed all = 0.389489941 MB/sec (with seq num) write speed all = 0.372687909 MB/sec (without seq num) 2024/11/23-18:03:21.019433 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273780: 925131 keys, 38448796 bytes, temperature: kWarm 2024/11/23-18:03:21.019504 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356201019468, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273780, "file_size": 38448796, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4050411210, "largest_seqno": 4167926108, "table_properties": {"data_size": 32479328, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903668, "raw_average_key_size": 28, "raw_value_size": 108314082, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925131, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356197, "oldest_key_time": 0, "file_creation_time": 1732356197, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273780, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:03:21.056867 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273781.sst) /dev/shm/ToplingTemp/Topling-142732-BP3bZU 2024/11/23-18:03:22.013158 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273781.sst): first pass time = 0.96's, 140.358'MB/sec 2024/11/23-18:03:23.376380 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273781.sst): old prealloc_size = 147639500, real_size = 34303840 2024/11/23-18:03:23.376947 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273781.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930929 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.18 avg-zval = 34.68 2024/11/23-18:03:23.377739 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273781.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:03:24.228448 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273781.sst): second pass time = 1.36's, 79.408'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.952'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3295.785'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 301.545'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33295.785'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.408'MB/sec zip pipeline throughput = 79.408'MB/sec entries = 930929 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.18 avg-zval = 34.68 usrkeys = 930929 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.18 avg-zval = 34.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930929 bytes = 3141904 } seq expand size = 6516503 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4458 value = 3.3495 dict = 1.00 all = 3.5043 } Zip/UnZip{ index = 0.0957 value = 0.2985 dict = 1.00 all = 0.2854 } ---------------------------- total value len = 11.015621 GB avg = 0.157 KB (by entry num) total key len = 1.934951 GB avg = 0.028 KB total ukey len = 1.350828 GB avg = 0.016 KB total ukey num = 0.083197930 Billion total entry num = 0.070005659 Billion write speed all = 0.393729970 MB/sec (with seq num) write speed all = 0.376703184 MB/sec (without seq num) 2024/11/23-18:03:24.281340 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273781: 930929 keys, 38300956 bytes, temperature: kWarm 2024/11/23-18:03:24.281406 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356204281370, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273781, "file_size": 38300956, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4076718630, "largest_seqno": 4167925128, "table_properties": {"data_size": 32288528, "index_size": 1782400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26066012, "raw_average_key_size": 28, "raw_value_size": 108151850, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930929, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356201, "oldest_key_time": 0, "file_creation_time": 1732356201, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273781, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:03:24.329733 139957500515904 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273782.sst) /dev/shm/ToplingTemp/Topling-142732-SmMOtU 2024/11/23-18:03:24.598471 139957500515904 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273782.sst): first pass time = 0.27's, 247.739'MB/sec 2024/11/23-18:03:25.231715 139957500515904 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273782.sst): old prealloc_size = 147639500, real_size = 16706096 2024/11/23-18:03:25.232034 139957500515904 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273782.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 461791 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0536 GB zip-val = 0.0157 GB avg-val = 116.15 avg-zval = 34.01 2024/11/23-18:03:25.232427 139957500515904 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273782.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/11/23-18:03:25.657664 139957500515904 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00/273782.sst): second pass time = 0.63's, 84.802'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.43's, 43.616'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2970.623'MB/sec (index lex order gen) rebuild zvType time = 0.36's, 0.319'MB/sec write SST data time = 0.06's, 291.282'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.541'MB, throughput = 14627.712'MB/sec zip my value time = 0.63's, unzip length = 0.054'GB zip my value throughput = 84.802'MB/sec zip pipeline throughput = 84.802'MB/sec entries = 461791 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.15 avg-zval = 34.01 usrkeys = 461791 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.15 avg-zval = 34.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 461791 bytes = 1443104 } seq expand size = 3232537 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0092 GB value = 0.0536 GB dict = 0.54 MB all = 0.0666 GB } __ZipSize{ index = 0.0009 GB value = 0.0157 GB dict = 0.54 MB all = 0.0186 GB } UnZip/Zip{ index = 10.4383 value = 3.4150 dict = 1.00 all = 3.5832 } Zip/UnZip{ index = 0.0958 value = 0.2928 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 11.072489 GB avg = 0.157 KB (by entry num) total key len = 1.947881 GB avg = 0.028 KB total ukey len = 1.360064 GB avg = 0.016 KB total ukey num = 0.083659721 Billion total entry num = 0.070467450 Billion write speed all = 0.395834814 MB/sec (with seq num) write speed all = 0.378696456 MB/sec (without seq num) 2024/11/23-18:03:25.687119 139957500515904 [compaction_job.cc:1948] [default] [JOB 32608] Generated table #273782: 461791 keys, 18577095 bytes, temperature: kWarm 2024/11/23-18:03:25.687188 139957500515904 EVENT_LOG_v1 {"time_micros": 1732356205687152, "cf_name": "default", "job": 32608, "event": "table_file_creation", "file_number": 273782, "file_size": 18577095, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4138066802, "largest_seqno": 4167925347, "table_properties": {"data_size": 15705696, "index_size": 884800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12930148, "raw_average_key_size": 28, "raw_value_size": 53635602, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 461791, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356204, "oldest_key_time": 0, "file_creation_time": 1732356204, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273782, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:03:25.706039 139957500515904 [compaction_job.cc:703] job-32608: subcompact[0], size: 0.745350 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32608/att-00 [273763,273764,273765,273766,273767,273768,273769,273770,273771,273772,273773,273774,273775,273776,273777,273778,273779,273780,273781,273782] 2024/11/23-18:03:25.826956 139957500515904 (Original Log Time 2024/11/23-18:02:16.373562) EVENT_LOG_v1 {"time_micros": 1732356136373533, "job": 32608, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [273734], "files_L3": [273754, 273755, 273683, 273685, 273686, 273687, 273688, 273689, 273690, 273691, 273692, 273693, 273694, 273695, 273696, 273697, 273698, 273699, 273700, 273701, 273756], "score": 0.99511, "input_data_size": 762797924, "oldest_snapshot_seqno": 4168166064} 2024/11/23-18:03:25.826960 139957500515904 (Original Log Time 2024/11/23-18:03:25.825311) [compaction_job.cc:2022] [default] [JOB 32608] Compacted 1@2 + 21@3 files to L3 => 745350293 bytes 2024/11/23-18:03:25.826962 139957500515904 (Original Log Time 2024/11/23-18:03:25.826881) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.0 rd, 10.8 wr, level 3, files in(1, 21) out(20 +0 blob) MB in(27.1, 700.3 +0.0 blob) out(710.8 +0.0 blob), read-write-amplify(53.0) write-amplify(26.2) OK, records in: 18189551, records dropped: 80265 output_compression: Snappy 2024/11/23-18:03:25.826965 139957500515904 (Original Log Time 2024/11/23-18:03:25.826930) EVENT_LOG_v1 {"time_micros": 1732356205826908, "cf": "default", "job": 32608, "event": "compaction_finished", "compaction_time_micros": 69332487, "compaction_time_cpu_micros": 28649399, "output_level": 3, "num_output_files": 20, "total_output_size": 745350293, "num_input_records": 18189551, "num_output_records": 18109286, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 551646714, "file_range_sync_nanos": 0, "file_fsync_nanos": 14622, "file_prepare_write_nanos": 54080, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-18:03:25.972551 139957500515904 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732356136,"executesMs":69585,"compactionJobId":32608,"attempt":0,"compactionInputRawBytes":2628101330,"compactionInputZipBytes":762797924,"compactionOutputRawBytes":0,"compactionOutputZipBytes":745350293,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-dcb681247794b080975a1b02da9035e0-94dfc1f12fec15ff-00"}}, response = 2024/11/23-18:03:25.972631 139957500515904 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32608/att-00: olev 3, work 69.586 s, result 7.741 ms, install 1.669 ms, input{raw 2.448 GiB zip 727.461 MiB}