2024/11/23-17:52:29.734103 139957492123200 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00: fork to child time = 0.006366 sec 2024/11/23-17:52:29.734778 139957492123200 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00 2024/11/23-17:52:29.741622 139957492123200 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00 2024/11/23-17:52:29.742522 139957492123200 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:52:29.742728 139957492123200 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:52:29.792296 139957492123200 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32417/att-00/MANIFEST-206347 2024/11/23-17:52:29.807889 139957492123200 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32417/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 273355, last_sequence is 4162351499, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 273169 2024/11/23-17:52:29.807894 139957492123200 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:52:29.808066 139957492123200 [version_set.cc:5721] Creating manifest 273355 2024/11/23-17:52:29.844513 139957492123200 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:52:29.865413 139957492123200 [compaction_job.cc:2437] [default] [JOB 32417] Compacting 1@2 + 21@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-17:52:29.865428 139957492123200 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24266 Base level 2, inputs: [273304(27MB)], [273326(5221KB) 273247(39MB) 273248(37MB) 273249(36MB) 273250(35MB) 273251(36MB) 273252(37MB) 273253(37MB) 273254(35MB) 273255(34MB) 273256(37MB) 273257(32MB) 273258(37MB) 273259(34MB) 273260(34MB) 273261(36MB) 273262(37MB) 273269(36MB) 273270(37MB) 273271(37MB) 273327(34MB)] 2024/11/23-17:52:29.872054 139957492123200 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:52:29.872079 139957492123200 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:52:29.872094 139957492123200 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:52:29.872099 139957492123200 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:52:29.872110 139957492123200 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:52:30.161019 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273356.sst) /dev/shm/ToplingTemp/Topling-139962-G9tTsv 2024/11/23-17:52:31.204334 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273356.sst): first pass time = 1.04's, 128.651'MB/sec 2024/11/23-17:52:32.937292 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273356.sst): old prealloc_size = 147639500, real_size = 36955488 2024/11/23-17:52:32.937948 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273356.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930794 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0349 GB avg-val = 116.20 avg-zval = 37.54 2024/11/23-17:52:32.938806 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273356.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:52:33.755830 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273356.sst): second pass time = 1.73's, 62.448'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 50.458'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2837.653'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 316.827'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37828.265'MB/sec zip my value time = 1.73's, unzip length = 0.108'GB zip my value throughput = 62.448'MB/sec zip pipeline throughput = 62.448'MB/sec entries = 930794 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.20 avg-zval = 37.54 usrkeys = 930794 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.20 avg-zval = 37.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930794 bytes = 3490496 } seq expand size = 6515558 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0413 GB } UnZip/Zip{ index = 10.4454 value = 3.0954 dict = 1.00 all = 3.2497 } Zip/UnZip{ index = 0.0957 value = 0.3231 dict = 1.00 all = 0.3077 } ---------------------------- total value len = 8.963291 GB avg = 0.168 KB (by entry num) total key len = 1.466883 GB avg = 0.028 KB total ukey len = 1.016494 GB avg = 0.015 KB total ukey num = 0.066481229 Billion total entry num = 0.053288958 Billion write speed all = 0.323501108 MB/sec (with seq num) write speed all = 0.310278673 MB/sec (without seq num) 2024/11/23-17:52:33.807532 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273356: 930794 keys, 41301310 bytes, temperature: kWarm 2024/11/23-17:52:33.807627 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355553807570, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273356, "file_size": 41301310, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4161868116, "table_properties": {"data_size": 34940368, "index_size": 1782208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26062232, "raw_average_key_size": 28, "raw_value_size": 108155624, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930794, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355550, "oldest_key_time": 0, "file_creation_time": 1732355550, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273356, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:52:33.846452 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273357.sst) /dev/shm/ToplingTemp/Topling-139962-Xejs7q 2024/11/23-17:52:34.792931 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273357.sst): first pass time = 0.95's, 141.813'MB/sec 2024/11/23-17:52:36.196043 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273357.sst): old prealloc_size = 147639500, real_size = 36068528 2024/11/23-17:52:36.196624 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273357.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928220 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0340 GB avg-val = 116.60 avg-zval = 36.68 2024/11/23-17:52:36.197244 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273357.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:52:36.992111 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273357.sst): second pass time = 1.40's, 77.204'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 49.496'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.730'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.352'MB/sec write SST data time = 0.14's, 289.761'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35137.714'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.204'MB/sec zip pipeline throughput = 77.204'MB/sec entries = 928220 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.60 avg-zval = 36.68 usrkeys = 928220 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.28 avg-zval = 36.68 TagRS{ kinds = RS_Key0_TagN bytes = 145120 } TagArray{ size = 621428 bytes = 2330368 } seq expand size = 4349996 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.3759 value = 3.1788 dict = 1.00 all = 3.4065 } Zip/UnZip{ index = 0.0964 value = 0.3146 dict = 1.00 all = 0.2936 } ---------------------------- total value len = 9.075869 GB avg = 0.167 KB (by entry num) total key len = 1.492873 GB avg = 0.028 KB total ukey len = 1.035058 GB avg = 0.015 KB total ukey num = 0.067409449 Billion total entry num = 0.054217178 Billion write speed all = 0.327766010 MB/sec (with seq num) write speed all = 0.314314609 MB/sec (without seq num) 2024/11/23-17:52:37.040193 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273357: 928220 keys, 39401024 bytes, temperature: kWarm 2024/11/23-17:52:37.040262 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355557040225, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273357, "file_size": 39401024, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 34047136, "index_size": 1789184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25990160, "raw_average_key_size": 28, "raw_value_size": 108227616, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928220, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355553, "oldest_key_time": 0, "file_creation_time": 1732355553, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273357, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:52:37.077114 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273358.sst) /dev/shm/ToplingTemp/Topling-139962-PRWo5q 2024/11/23-17:52:38.116215 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273358.sst): first pass time = 1.04's, 129.171'MB/sec 2024/11/23-17:52:39.480243 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273358.sst): old prealloc_size = 147639500, real_size = 34150928 2024/11/23-17:52:39.480830 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273358.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931491 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0321 GB avg-val = 116.09 avg-zval = 34.50 2024/11/23-17:52:39.482545 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273358.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:52:40.329708 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273358.sst): second pass time = 1.36's, 79.327'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.044'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3155.011'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 293.254'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32390.830'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.327'MB/sec zip pipeline throughput = 79.327'MB/sec entries = 931491 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.09 avg-zval = 34.50 usrkeys = 931491 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.09 avg-zval = 34.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931491 bytes = 3260240 } seq expand size = 6520437 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4450 value = 3.3651 dict = 1.00 all = 3.5078 } Zip/UnZip{ index = 0.0957 value = 0.2972 dict = 1.00 all = 0.2851 } ---------------------------- total value len = 9.190525 GB avg = 0.167 KB (by entry num) total key len = 1.518955 GB avg = 0.028 KB total ukey len = 1.053688 GB avg = 0.015 KB total ukey num = 0.068340940 Billion total entry num = 0.055148669 Billion write speed all = 0.332096321 MB/sec (with seq num) write speed all = 0.318415231 MB/sec (without seq num) 2024/11/23-17:52:40.383505 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273358: 931491 keys, 38262208 bytes, temperature: kWarm 2024/11/23-17:52:40.383576 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355560383539, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273358, "file_size": 38262208, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775902, "table_properties": {"data_size": 32134272, "index_size": 1783616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26081748, "raw_average_key_size": 28, "raw_value_size": 108136122, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931491, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355557, "oldest_key_time": 0, "file_creation_time": 1732355557, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273358, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:52:40.421378 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273359.sst) /dev/shm/ToplingTemp/Topling-139962-palAar 2024/11/23-17:52:41.386321 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273359.sst): first pass time = 0.96's, 139.100'MB/sec 2024/11/23-17:52:42.681222 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273359.sst): old prealloc_size = 147639500, real_size = 32453968 2024/11/23-17:52:42.681799 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273359.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.09 avg-zval = 32.69 2024/11/23-17:52:42.682614 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273359.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:52:43.534066 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273359.sst): second pass time = 1.29's, 83.560'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.148'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3187.315'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 283.477'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33970.255'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.560'MB/sec zip pipeline throughput = 83.560'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 32.69 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 32.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5122 value = 3.5514 dict = 1.00 all = 3.6474 } Zip/UnZip{ index = 0.0951 value = 0.2816 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 9.305182 GB avg = 0.166 KB (by entry num) total key len = 1.545036 GB avg = 0.028 KB total ukey len = 1.072317 GB avg = 0.015 KB total ukey num = 0.069272402 Billion total entry num = 0.056080131 Billion write speed all = 0.336427116 MB/sec (with seq num) write speed all = 0.322516335 MB/sec (without seq num) 2024/11/23-17:52:43.582505 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273359: 931462 keys, 36798032 bytes, temperature: kWarm 2024/11/23-17:52:43.582576 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355563582538, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273359, "file_size": 36798032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 30448768, "index_size": 1772160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355560, "oldest_key_time": 0, "file_creation_time": 1732355560, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273359, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:52:43.632147 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273360.sst) /dev/shm/ToplingTemp/Topling-139962-IGynQt 2024/11/23-17:52:44.639851 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273360.sst): first pass time = 1.01's, 133.197'MB/sec 2024/11/23-17:52:45.976782 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273360.sst): old prealloc_size = 147639500, real_size = 33818416 2024/11/23-17:52:45.977347 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273360.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0318 GB avg-val = 117.04 avg-zval = 34.39 2024/11/23-17:52:45.978147 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273360.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:52:46.819346 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273360.sst): second pass time = 1.34's, 81.062'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.134'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3249.734'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 288.919'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32741.780'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.062'MB/sec zip pipeline throughput = 81.062'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 34.39 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 34.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3354608 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5137 value = 3.4030 dict = 1.00 all = 3.5294 } Zip/UnZip{ index = 0.0951 value = 0.2939 dict = 1.00 all = 0.2833 } ---------------------------- total value len = 9.419966 GB avg = 0.165 KB (by entry num) total key len = 1.570947 GB avg = 0.028 KB total ukey len = 1.090825 GB avg = 0.016 KB total ukey num = 0.070197807 Billion total entry num = 0.057005536 Billion write speed all = 0.340754881 MB/sec (with seq num) write speed all = 0.326615992 MB/sec (without seq num) 2024/11/23-17:52:46.866471 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273360: 925405 keys, 38027982 bytes, temperature: kWarm 2024/11/23-17:52:46.866544 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355566866504, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273360, "file_size": 38027982, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775947, "table_properties": {"data_size": 31826528, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355563, "oldest_key_time": 0, "file_creation_time": 1732355563, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273360, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:52:46.909960 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273361.sst) /dev/shm/ToplingTemp/Topling-139962-qZfxsr 2024/11/23-17:52:47.869731 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273361.sst): first pass time = 0.96's, 139.852'MB/sec 2024/11/23-17:52:49.234730 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273361.sst): old prealloc_size = 147639500, real_size = 35861904 2024/11/23-17:52:49.235287 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273361.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924151 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0339 GB avg-val = 117.23 avg-zval = 36.64 2024/11/23-17:52:49.236073 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273361.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:52:50.070671 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273361.sst): second pass time = 1.36's, 79.445'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.934'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3308.973'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 305.769'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 29349.661'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.445'MB/sec zip pipeline throughput = 79.445'MB/sec entries = 924151 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.23 avg-zval = 36.64 usrkeys = 924151 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.23 avg-zval = 36.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924151 bytes = 3350064 } seq expand size = 6469057 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.4451 value = 3.1996 dict = 1.00 all = 3.3497 } Zip/UnZip{ index = 0.0957 value = 0.3125 dict = 1.00 all = 0.2985 } ---------------------------- total value len = 9.534777 GB avg = 0.165 KB (by entry num) total key len = 1.596824 GB avg = 0.028 KB total ukey len = 1.109308 GB avg = 0.016 KB total ukey num = 0.071121958 Billion total entry num = 0.057929687 Billion write speed all = 0.345081856 MB/sec (with seq num) write speed all = 0.330715201 MB/sec (without seq num) 2024/11/23-17:52:50.120612 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273361: 924151 keys, 40068826 bytes, temperature: kWarm 2024/11/23-17:52:50.120684 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355570120646, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273361, "file_size": 40068826, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33861120, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876228, "raw_average_key_size": 28, "raw_value_size": 108341550, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924151, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355566, "oldest_key_time": 0, "file_creation_time": 1732355566, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273361, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:52:50.160201 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273362.sst) /dev/shm/ToplingTemp/Topling-139962-7VZKyt 2024/11/23-17:52:51.121353 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273362.sst): first pass time = 0.96's, 139.649'MB/sec 2024/11/23-17:52:52.491253 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273362.sst): old prealloc_size = 147639500, real_size = 34913904 2024/11/23-17:52:52.491837 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273362.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930745 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0329 GB avg-val = 116.20 avg-zval = 35.35 2024/11/23-17:52:52.492570 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273362.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:52:53.313679 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273362.sst): second pass time = 1.37's, 79.004'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 47.080'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3171.805'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 292.064'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29851.827'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.004'MB/sec zip pipeline throughput = 79.004'MB/sec entries = 930745 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.20 avg-zval = 35.35 usrkeys = 930745 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.83 avg-zval = 35.35 TagRS{ kinds = RS_Key0_TagN bytes = 145520 } TagArray{ size = 747748 bytes = 2804064 } seq expand size = 5234236 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.4464 value = 3.2875 dict = 1.00 all = 3.4665 } Zip/UnZip{ index = 0.0957 value = 0.3042 dict = 1.00 all = 0.2885 } ---------------------------- total value len = 9.648168 GB avg = 0.164 KB (by entry num) total key len = 1.622885 GB avg = 0.028 KB total ukey len = 1.127923 GB avg = 0.016 KB total ukey num = 0.072052703 Billion total entry num = 0.058860432 Billion write speed all = 0.349369775 MB/sec (with seq num) write speed all = 0.334773761 MB/sec (without seq num) 2024/11/23-17:52:53.361937 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273362: 930745 keys, 38718776 bytes, temperature: kWarm 2024/11/23-17:52:53.362021 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355573361984, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273362, "file_size": 38718776, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 32899104, "index_size": 1781952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26060860, "raw_average_key_size": 28, "raw_value_size": 108156994, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930745, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355570, "oldest_key_time": 0, "file_creation_time": 1732355570, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273362, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:52:53.518378 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273363.sst) /dev/shm/ToplingTemp/Topling-139962-Bat6Yt 2024/11/23-17:52:54.391522 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273363.sst): first pass time = 0.87's, 153.727'MB/sec 2024/11/23-17:52:55.740082 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273363.sst): old prealloc_size = 147639500, real_size = 32632016 2024/11/23-17:52:55.740620 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273363.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.49 avg-zval = 32.98 2024/11/23-17:52:55.741438 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273363.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:52:56.587377 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273363.sst): second pass time = 1.35's, 80.288'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.633'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3416.670'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 287.162'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33592.570'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.288'MB/sec zip pipeline throughput = 80.288'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 32.98 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3483408 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5137 value = 3.5325 dict = 1.00 all = 3.6306 } Zip/UnZip{ index = 0.0951 value = 0.2831 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 9.762879 GB avg = 0.163 KB (by entry num) total key len = 1.648894 GB avg = 0.028 KB total ukey len = 1.146501 GB avg = 0.016 KB total ukey num = 0.072981608 Billion total entry num = 0.059789337 Billion write speed all = 0.353695799 MB/sec (with seq num) write speed all = 0.338870942 MB/sec (without seq num) 2024/11/23-17:52:56.634718 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273363: 928905 keys, 36968638 bytes, temperature: kWarm 2024/11/23-17:52:56.634793 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355576634752, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273363, "file_size": 36968638, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3283075579, "largest_seqno": 4012776210, "table_properties": {"data_size": 30632576, "index_size": 1767040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355573, "oldest_key_time": 0, "file_creation_time": 1732355573, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273363, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:52:56.678460 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273364.sst) /dev/shm/ToplingTemp/Topling-139962-gWSrdu 2024/11/23-17:52:57.687118 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273364.sst): first pass time = 1.01's, 133.072'MB/sec 2024/11/23-17:52:58.972189 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273364.sst): old prealloc_size = 147639500, real_size = 32453424 2024/11/23-17:52:58.972738 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273364.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931692 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.06 avg-zval = 32.68 2024/11/23-17:52:58.973576 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273364.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:52:59.825345 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273364.sst): second pass time = 1.28's, 84.199'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.997'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3354.174'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 285.926'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35387.908'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.199'MB/sec zip pipeline throughput = 84.199'MB/sec entries = 931692 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 32.68 usrkeys = 931692 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 32.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931692 bytes = 3377392 } seq expand size = 6521844 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5136 value = 3.5513 dict = 1.00 all = 3.6590 } Zip/UnZip{ index = 0.0951 value = 0.2816 dict = 1.00 all = 0.2733 } ---------------------------- total value len = 9.877531 GB avg = 0.163 KB (by entry num) total key len = 1.674981 GB avg = 0.028 KB total ukey len = 1.165135 GB avg = 0.016 KB total ukey num = 0.073913300 Billion total entry num = 0.060721029 Billion write speed all = 0.358021944 MB/sec (with seq num) write speed all = 0.342967584 MB/sec (without seq num) 2024/11/23-17:52:59.870937 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273364: 931692 keys, 36681856 bytes, temperature: kWarm 2024/11/23-17:52:59.871019 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355579870972, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273364, "file_size": 36681856, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776228, "table_properties": {"data_size": 30447968, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26087376, "raw_average_key_size": 28, "raw_value_size": 108130408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931692, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355576, "oldest_key_time": 0, "file_creation_time": 1732355576, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273364, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:52:59.913301 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273365.sst) /dev/shm/ToplingTemp/Topling-139962-Xucxdv 2024/11/23-17:53:00.979924 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273365.sst): first pass time = 1.07's, 125.840'MB/sec 2024/11/23-17:53:02.338413 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273365.sst): old prealloc_size = 147639500, real_size = 33978816 2024/11/23-17:53:02.338999 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273365.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933726 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0320 GB avg-val = 115.74 avg-zval = 34.23 2024/11/23-17:53:02.339818 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273365.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:53:03.190874 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273365.sst): second pass time = 1.36's, 79.601'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.829'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.340'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 296.028'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33013.109'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.601'MB/sec zip pipeline throughput = 79.601'MB/sec entries = 933726 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.74 avg-zval = 34.23 usrkeys = 933726 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.74 avg-zval = 34.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933726 bytes = 3384768 } seq expand size = 6536082 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4438 value = 3.3818 dict = 1.00 all = 3.5123 } Zip/UnZip{ index = 0.0958 value = 0.2957 dict = 1.00 all = 0.2847 } ---------------------------- total value len = 9.992141 GB avg = 0.162 KB (by entry num) total key len = 1.701126 GB avg = 0.028 KB total ukey len = 1.183810 GB avg = 0.016 KB total ukey num = 0.074847026 Billion total entry num = 0.061654755 Billion write speed all = 0.362346232 MB/sec (with seq num) write speed all = 0.347061970 MB/sec (without seq num) 2024/11/23-17:53:03.242778 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273365: 933726 keys, 38213958 bytes, temperature: kWarm 2024/11/23-17:53:03.242847 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355583242811, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273365, "file_size": 38213958, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 4012776268, "table_properties": {"data_size": 31957104, "index_size": 1788096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26144328, "raw_average_key_size": 28, "raw_value_size": 108073524, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933726, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355579, "oldest_key_time": 0, "file_creation_time": 1732355579, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273365, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:03.282101 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273366.sst) /dev/shm/ToplingTemp/Topling-139962-K7H9at 2024/11/23-17:53:04.274707 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273366.sst): first pass time = 0.99's, 135.222'MB/sec 2024/11/23-17:53:05.561900 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273366.sst): old prealloc_size = 147639500, real_size = 30743024 2024/11/23-17:53:05.562518 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273366.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928070 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0287 GB avg-val = 116.62 avg-zval = 30.96 2024/11/23-17:53:05.563353 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273366.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:53:06.384716 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273366.sst): second pass time = 1.29's, 84.215'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.631'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2982.186'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 265.896'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34659.177'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.215'MB/sec zip pipeline throughput = 84.215'MB/sec entries = 928070 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.62 avg-zval = 30.96 usrkeys = 928070 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.62 avg-zval = 30.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928070 bytes = 3480272 } seq expand size = 6496490 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4433 value = 3.7667 dict = 1.00 all = 3.8264 } Zip/UnZip{ index = 0.0958 value = 0.2655 dict = 1.00 all = 0.2613 } ---------------------------- total value len = 10.106869 GB avg = 0.161 KB (by entry num) total key len = 1.727112 GB avg = 0.028 KB total ukey len = 1.202371 GB avg = 0.016 KB total ukey num = 0.075775096 Billion total entry num = 0.062582825 Billion write speed all = 0.366670338 MB/sec (with seq num) write speed all = 0.351157543 MB/sec (without seq num) 2024/11/23-17:53:06.432641 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273366: 928070 keys, 35076844 bytes, temperature: kWarm 2024/11/23-17:53:06.432710 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355586432675, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273366, "file_size": 35076844, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 28733472, "index_size": 1777344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25985960, "raw_average_key_size": 28, "raw_value_size": 108231768, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928070, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355583, "oldest_key_time": 0, "file_creation_time": 1732355583, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273366, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:06.475297 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273367.sst) /dev/shm/ToplingTemp/Topling-139962-aXHfiv 2024/11/23-17:53:07.468727 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273367.sst): first pass time = 0.99's, 135.111'MB/sec 2024/11/23-17:53:08.809918 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273367.sst): old prealloc_size = 147639500, real_size = 35166896 2024/11/23-17:53:08.810503 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273367.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926283 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 116.90 avg-zval = 35.81 2024/11/23-17:53:08.811345 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273367.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:53:09.649541 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273367.sst): second pass time = 1.34's, 80.792'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.040'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3136.336'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 306.029'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33326.071'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.792'MB/sec zip pipeline throughput = 80.792'MB/sec entries = 926283 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.90 avg-zval = 35.81 usrkeys = 926283 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.90 avg-zval = 35.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926283 bytes = 3473584 } seq expand size = 6483981 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5141 value = 3.2641 dict = 1.00 all = 3.3984 } Zip/UnZip{ index = 0.0951 value = 0.3064 dict = 1.00 all = 0.2943 } ---------------------------- total value len = 10.221635 GB avg = 0.161 KB (by entry num) total key len = 1.753048 GB avg = 0.028 KB total ukey len = 1.220897 GB avg = 0.016 KB total ukey num = 0.076701379 Billion total entry num = 0.063509108 Billion write speed all = 0.370992387 MB/sec (with seq num) write speed all = 0.355251580 MB/sec (without seq num) 2024/11/23-17:53:09.700999 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273367: 926283 keys, 39494634 bytes, temperature: kWarm 2024/11/23-17:53:09.701070 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355589701035, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273367, "file_size": 39494634, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33173152, "index_size": 1761984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25935924, "raw_average_key_size": 28, "raw_value_size": 108281842, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926283, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355586, "oldest_key_time": 0, "file_creation_time": 1732355586, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273367, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:09.738460 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273368.sst) /dev/shm/ToplingTemp/Topling-139962-EbOWiu 2024/11/23-17:53:10.763345 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273368.sst): first pass time = 1.02's, 130.964'MB/sec 2024/11/23-17:53:12.074803 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273368.sst): old prealloc_size = 147639500, real_size = 31089776 2024/11/23-17:53:12.075385 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273368.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0291 GB avg-val = 116.15 avg-zval = 31.23 2024/11/23-17:53:12.076219 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273368.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:53:12.906101 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273368.sst): second pass time = 1.31's, 82.513'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.625'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3210.597'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 275.764'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33616.907'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.513'MB/sec zip pipeline throughput = 82.513'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 31.23 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 31.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4445 value = 3.7197 dict = 1.00 all = 3.7879 } Zip/UnZip{ index = 0.0957 value = 0.2688 dict = 1.00 all = 0.2640 } ---------------------------- total value len = 10.336300 GB avg = 0.160 KB (by entry num) total key len = 1.779118 GB avg = 0.028 KB total ukey len = 1.239519 GB avg = 0.016 KB total ukey num = 0.077632462 Billion total entry num = 0.064440191 Billion write speed all = 0.375314700 MB/sec (with seq num) write speed all = 0.359344735 MB/sec (without seq num) 2024/11/23-17:53:12.953451 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273368: 931083 keys, 35432912 bytes, temperature: kWarm 2024/11/23-17:53:12.953521 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355592953484, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273368, "file_size": 35432912, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3419724704, "largest_seqno": 4012776344, "table_properties": {"data_size": 29073888, "index_size": 1782912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355589, "oldest_key_time": 0, "file_creation_time": 1732355589, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273368, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:12.990024 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273369.sst) /dev/shm/ToplingTemp/Topling-139962-WSxpgr 2024/11/23-17:53:14.001958 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273369.sst): first pass time = 1.01's, 132.641'MB/sec 2024/11/23-17:53:15.308781 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273369.sst): old prealloc_size = 147639500, real_size = 31077568 2024/11/23-17:53:15.309381 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273369.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924809 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0291 GB avg-val = 117.13 avg-zval = 31.41 2024/11/23-17:53:15.310205 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273369.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:53:16.127448 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273369.sst): second pass time = 1.31's, 82.942'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.101'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3095.903'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 276.476'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36038.592'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.942'MB/sec zip pipeline throughput = 82.942'MB/sec entries = 924809 avg-key = 28.00 avg-zkey = 1.94 avg-val = 117.13 avg-zval = 31.41 usrkeys = 924809 avg-key = 20.00 avg-zkey = 1.94 avg-val = 124.13 avg-zval = 31.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924809 bytes = 3352448 } seq expand size = 6473663 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.3086 value = 3.7286 dict = 1.00 all = 3.8039 } Zip/UnZip{ index = 0.0970 value = 0.2682 dict = 1.00 all = 0.2629 } ---------------------------- total value len = 10.451097 GB avg = 0.160 KB (by entry num) total key len = 1.805012 GB avg = 0.028 KB total ukey len = 1.258015 GB avg = 0.016 KB total ukey num = 0.078557271 Billion total entry num = 0.065365000 Billion write speed all = 0.379635193 MB/sec (with seq num) write speed all = 0.363437652 MB/sec (without seq num) 2024/11/23-17:53:16.173541 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273369: 924809 keys, 35284142 bytes, temperature: kWarm 2024/11/23-17:53:16.173611 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355596173574, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273369, "file_size": 35284142, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3531327642, "largest_seqno": 4012776360, "table_properties": {"data_size": 29051952, "index_size": 1794240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25894652, "raw_average_key_size": 28, "raw_value_size": 108323106, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924809, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355592, "oldest_key_time": 0, "file_creation_time": 1732355592, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273369, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:16.212067 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273370.sst) /dev/shm/ToplingTemp/Topling-139962-OE761q 2024/11/23-17:53:17.094307 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273370.sst): first pass time = 0.88's, 152.140'MB/sec 2024/11/23-17:53:18.610717 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273370.sst): old prealloc_size = 147639500, real_size = 36476976 2024/11/23-17:53:18.611283 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273370.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927102 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.19 2024/11/23-17:53:18.611429 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273370.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:53:19.305260 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273370.sst): second pass time = 1.52's, 71.429'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 53.992'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3242.573'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.407'MB/sec write SST data time = 0.12's, 302.803'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32688.249'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.429'MB/sec zip pipeline throughput = 71.429'MB/sec entries = 927102 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 37.19 usrkeys = 927102 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 37.19 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 6745 bytes = 17728 } seq expand size = 47215 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5123 value = 3.1397 dict = 1.00 all = 3.5792 } Zip/UnZip{ index = 0.0951 value = 0.3185 dict = 1.00 all = 0.2794 } ---------------------------- total value len = 10.559403 GB avg = 0.159 KB (by entry num) total key len = 1.830971 GB avg = 0.028 KB total ukey len = 1.276557 GB avg = 0.016 KB total ukey num = 0.079484373 Billion total entry num = 0.066292102 Billion write speed all = 0.383756300 MB/sec (with seq num) write speed all = 0.367330638 MB/sec (without seq num) 2024/11/23-17:53:19.352560 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273370: 927102 keys, 37498859 bytes, temperature: kWarm 2024/11/23-17:53:19.352628 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355599352594, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273370, "file_size": 37498859, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012773626, "table_properties": {"data_size": 34481184, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958856, "raw_average_key_size": 28, "raw_value_size": 108258904, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927102, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355596, "oldest_key_time": 0, "file_creation_time": 1732355596, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273370, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:19.389827 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273371.sst) /dev/shm/ToplingTemp/Topling-139962-CnC4bu 2024/11/23-17:53:20.425900 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273371.sst): first pass time = 1.04's, 129.550'MB/sec 2024/11/23-17:53:21.858519 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273371.sst): old prealloc_size = 147639500, real_size = 37001024 2024/11/23-17:53:21.859083 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273371.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926511 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0350 GB avg-val = 116.86 avg-zval = 37.76 2024/11/23-17:53:21.859693 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273371.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:53:22.628725 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273371.sst): second pass time = 1.43's, 75.648'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 51.999'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3295.933'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.362'MB/sec write SST data time = 0.13's, 307.462'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36264.165'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.648'MB/sec zip pipeline throughput = 75.648'MB/sec entries = 926511 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.86 avg-zval = 37.76 usrkeys = 926511 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.98 avg-zval = 37.76 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 545275 bytes = 2044800 } seq expand size = 3816925 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0350 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.3798 value = 3.0950 dict = 1.00 all = 3.3513 } Zip/UnZip{ index = 0.0963 value = 0.3231 dict = 1.00 all = 0.2984 } ---------------------------- total value len = 10.671496 GB avg = 0.159 KB (by entry num) total key len = 1.856914 GB avg = 0.028 KB total ukey len = 1.295087 GB avg = 0.016 KB total ukey num = 0.080410884 Billion total entry num = 0.067218613 Billion write speed all = 0.387991593 MB/sec (with seq num) write speed all = 0.371338077 MB/sec (without seq num) 2024/11/23-17:53:22.678588 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273371: 926511 keys, 40048974 bytes, temperature: kWarm 2024/11/23-17:53:22.678660 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355602678624, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273371, "file_size": 40048974, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4161866011, "table_properties": {"data_size": 34983984, "index_size": 1785216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942308, "raw_average_key_size": 28, "raw_value_size": 108275526, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926511, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355599, "oldest_key_time": 0, "file_creation_time": 1732355599, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273371, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:22.717262 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273372.sst) /dev/shm/ToplingTemp/Topling-139962-fqC8pu 2024/11/23-17:53:23.757013 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273372.sst): first pass time = 1.04's, 129.091'MB/sec 2024/11/23-17:53:25.131340 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273372.sst): old prealloc_size = 147639500, real_size = 34992992 2024/11/23-17:53:25.131882 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273372.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927745 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0330 GB avg-val = 116.67 avg-zval = 35.57 2024/11/23-17:53:25.132713 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273372.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:53:25.969081 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273372.sst): second pass time = 1.37's, 78.804'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.529'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3475.369'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 303.443'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29867.714'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.804'MB/sec zip pipeline throughput = 78.804'MB/sec entries = 927745 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.67 avg-zval = 35.57 usrkeys = 927745 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.67 avg-zval = 35.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927745 bytes = 3131152 } seq expand size = 6494215 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.5131 value = 3.2804 dict = 1.00 all = 3.4434 } Zip/UnZip{ index = 0.0951 value = 0.3048 dict = 1.00 all = 0.2904 } ---------------------------- total value len = 10.786231 GB avg = 0.158 KB (by entry num) total key len = 1.882891 GB avg = 0.028 KB total ukey len = 1.313642 GB avg = 0.016 KB total ukey num = 0.081338629 Billion total entry num = 0.068146358 Billion write speed all = 0.392308712 MB/sec (with seq num) write speed all = 0.375427093 MB/sec (without seq num) 2024/11/23-17:53:26.017320 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273372: 927745 keys, 38978136 bytes, temperature: kWarm 2024/11/23-17:53:26.017399 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355606017356, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273372, "file_size": 38978136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4072856654, "largest_seqno": 4161868327, "table_properties": {"data_size": 32995920, "index_size": 1764928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976860, "raw_average_key_size": 28, "raw_value_size": 108240954, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927745, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355602, "oldest_key_time": 0, "file_creation_time": 1732355602, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273372, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:26.055157 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273373.sst) /dev/shm/ToplingTemp/Topling-139962-GgfT1u 2024/11/23-17:53:27.110396 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273373.sst): first pass time = 1.06's, 127.196'MB/sec 2024/11/23-17:53:28.449786 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273373.sst): old prealloc_size = 147639500, real_size = 34686592 2024/11/23-17:53:28.450375 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273373.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926796 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0327 GB avg-val = 116.82 avg-zval = 35.27 2024/11/23-17:53:28.451132 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273373.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:53:29.297398 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273373.sst): second pass time = 1.34's, 80.926'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.486'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3106.433'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 300.985'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34350.184'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.926'MB/sec zip pipeline throughput = 80.926'MB/sec entries = 926796 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.82 avg-zval = 35.27 usrkeys = 926796 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.82 avg-zval = 35.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926796 bytes = 3012096 } seq expand size = 6487572 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5142 value = 3.3118 dict = 1.00 all = 3.4814 } Zip/UnZip{ index = 0.0951 value = 0.3020 dict = 1.00 all = 0.2872 } ---------------------------- total value len = 10.900986 GB avg = 0.158 KB (by entry num) total key len = 1.908841 GB avg = 0.028 KB total ukey len = 1.332178 GB avg = 0.016 KB total ukey num = 0.082265425 Billion total entry num = 0.069073154 Billion write speed all = 0.396624882 MB/sec (with seq num) write speed all = 0.379515434 MB/sec (without seq num) 2024/11/23-17:53:29.345008 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273373: 926796 keys, 38553230 bytes, temperature: kWarm 2024/11/23-17:53:29.345077 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355609345041, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273373, "file_size": 38553230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4116739047, "largest_seqno": 4161868245, "table_properties": {"data_size": 32691760, "index_size": 1762944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25950288, "raw_average_key_size": 28, "raw_value_size": 108267548, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926796, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355606, "oldest_key_time": 0, "file_creation_time": 1732355606, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273373, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:29.381574 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273374.sst) /dev/shm/ToplingTemp/Topling-139962-wgYV0s 2024/11/23-17:53:30.423602 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273374.sst): first pass time = 1.04's, 128.809'MB/sec 2024/11/23-17:53:31.742427 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273374.sst): old prealloc_size = 147639500, real_size = 34390208 2024/11/23-17:53:31.742998 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273374.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925890 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 116.96 avg-zval = 34.99 2024/11/23-17:53:31.743762 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273374.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:53:32.594664 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273374.sst): second pass time = 1.32's, 82.194'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.024'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3210.053'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 297.621'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34340.473'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.194'MB/sec zip pipeline throughput = 82.194'MB/sec entries = 925890 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 34.99 usrkeys = 925890 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 34.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925890 bytes = 3124896 } seq expand size = 6481230 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5123 value = 3.3427 dict = 1.00 all = 3.4980 } Zip/UnZip{ index = 0.0951 value = 0.2992 dict = 1.00 all = 0.2859 } ---------------------------- total value len = 11.015760 GB avg = 0.157 KB (by entry num) total key len = 1.934766 GB avg = 0.028 KB total ukey len = 1.350696 GB avg = 0.016 KB total ukey num = 0.083191315 Billion total entry num = 0.069999044 Billion write speed all = 0.400940351 MB/sec (with seq num) write speed all = 0.383603331 MB/sec (without seq num) 2024/11/23-17:53:32.642782 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273374: 925890 keys, 38370114 bytes, temperature: kWarm 2024/11/23-17:53:32.642861 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355612642816, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273374, "file_size": 38370114, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4050411210, "largest_seqno": 4161868035, "table_properties": {"data_size": 32397040, "index_size": 1761536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924920, "raw_average_key_size": 28, "raw_value_size": 108292940, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925890, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355609, "oldest_key_time": 0, "file_creation_time": 1732355609, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273374, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:32.688654 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273375.sst) /dev/shm/ToplingTemp/Topling-139962-euwY2s 2024/11/23-17:53:33.707946 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273375.sst): first pass time = 1.02's, 131.683'MB/sec 2024/11/23-17:53:35.028089 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273375.sst): old prealloc_size = 147639500, real_size = 34526352 2024/11/23-17:53:35.028673 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273375.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928245 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.59 avg-zval = 35.03 2024/11/23-17:53:35.029442 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273375.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:53:35.873340 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273375.sst): second pass time = 1.32's, 82.031'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.569'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3174.740'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 299.566'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37393.641'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.031'MB/sec zip pipeline throughput = 82.031'MB/sec entries = 928245 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.59 avg-zval = 35.03 usrkeys = 928245 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.59 avg-zval = 35.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928245 bytes = 3132848 } seq expand size = 6497715 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.4457 value = 3.3283 dict = 1.00 all = 3.4847 } Zip/UnZip{ index = 0.0957 value = 0.3005 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 11.130485 GB avg = 0.157 KB (by entry num) total key len = 1.960757 GB avg = 0.028 KB total ukey len = 1.369261 GB avg = 0.016 KB total ukey num = 0.084119560 Billion total entry num = 0.070927289 Billion write speed all = 0.405255683 MB/sec (with seq num) write speed all = 0.387690542 MB/sec (without seq num) 2024/11/23-17:53:35.920425 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273375: 928245 keys, 38516328 bytes, temperature: kWarm 2024/11/23-17:53:35.920495 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355615920459, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273375, "file_size": 38516328, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4076028341, "largest_seqno": 4161868187, "table_properties": {"data_size": 32516800, "index_size": 1777280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25990860, "raw_average_key_size": 28, "raw_value_size": 108226966, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928245, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355612, "oldest_key_time": 0, "file_creation_time": 1732355612, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273375, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:35.975822 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273376.sst) /dev/shm/ToplingTemp/Topling-139962-zy6qXq 2024/11/23-17:53:36.135658 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273376.sst): first pass time = 0.16's, 236.745'MB/sec 2024/11/23-17:53:36.480940 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273376.sst): old prealloc_size = 147639500, real_size = 9973088 2024/11/23-17:53:36.481160 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273376.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 263832 prefix = 4 raw-key = 0.0042 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0304 GB zip-val = 0.0094 GB avg-val = 115.40 avg-zval = 35.63 2024/11/23-17:53:36.481416 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273376.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.038 GB 2024/11/23-17:53:36.720117 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00/273376.sst): second pass time = 0.34's, 88.353'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.24's, 46.180'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2914.401'MB/sec (index lex order gen) rebuild zvType time = 0.20's, 0.328'MB/sec write SST data time = 0.04's, 292.839'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.311'MB, throughput = 10189.764'MB/sec zip my value time = 0.34's, unzip length = 0.030'GB zip my value throughput = 88.353'MB/sec zip pipeline throughput = 88.353'MB/sec entries = 263832 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.40 avg-zval = 35.63 usrkeys = 263832 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.40 avg-zval = 35.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 263832 bytes = 824496 } seq expand size = 1846824 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0053 GB value = 0.0304 GB dict = 0.31 MB all = 0.0378 GB } __ZipSize{ index = 0.0005 GB value = 0.0094 GB dict = 0.31 MB all = 0.0110 GB } UnZip/Zip{ index = 10.4258 value = 3.2386 dict = 1.00 all = 3.4255 } Zip/UnZip{ index = 0.0959 value = 0.3088 dict = 1.00 all = 0.2919 } ---------------------------- total value len = 11.162777 GB avg = 0.157 KB (by entry num) total key len = 1.968144 GB avg = 0.028 KB total ukey len = 1.374537 GB avg = 0.016 KB total ukey num = 0.084383392 Billion total entry num = 0.071191121 Billion write speed all = 0.406473360 MB/sec (with seq num) write speed all = 0.388843343 MB/sec (without seq num) 2024/11/23-17:53:36.739043 139957492123200 [compaction_job.cc:1948] [default] [JOB 32417] Generated table #273376: 263832 keys, 11044381 bytes, temperature: kWarm 2024/11/23-17:53:36.739098 139957492123200 EVENT_LOG_v1 {"time_micros": 1732355616739069, "cf_name": "default", "job": 32417, "event": "table_file_creation", "file_number": 273376, "file_size": 11044381, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4134398447, "largest_seqno": 4161867308, "table_properties": {"data_size": 9400848, "index_size": 506112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7387296, "raw_average_key_size": 28, "raw_value_size": 30445496, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 263832, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355615, "oldest_key_time": 0, "file_creation_time": 1732355615, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273376, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:53:36.752240 139957492123200 [compaction_job.cc:703] job-32417: subcompact[0], size: 0.772741 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32417/att-00 [273356,273357,273358,273359,273360,273361,273362,273363,273364,273365,273366,273367,273368,273369,273370,273371,273372,273373,273374,273375,273376] 2024/11/23-17:53:36.875615 139957492123200 (Original Log Time 2024/11/23-17:52:29.865465) EVENT_LOG_v1 {"time_micros": 1732355549865437, "job": 32417, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [273304], "files_L3": [273326, 273247, 273248, 273249, 273250, 273251, 273252, 273253, 273254, 273255, 273256, 273257, 273258, 273259, 273260, 273261, 273262, 273269, 273270, 273271, 273327], "score": 0.990852, "input_data_size": 797875718, "oldest_snapshot_seqno": 4162351439} 2024/11/23-17:53:36.875623 139957492123200 (Original Log Time 2024/11/23-17:53:36.873188) [compaction_job.cc:2022] [default] [JOB 32417] Compacted 1@2 + 21@3 files to L3 => 772741164 bytes 2024/11/23-17:53:36.875624 139957492123200 (Original Log Time 2024/11/23-17:53:36.874682) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 11.9 rd, 11.6 wr, level 3, files in(1, 21) out(21 +0 blob) MB in(27.1, 733.8 +0.0 blob) out(736.9 +0.0 blob), read-write-amplify(55.3) write-amplify(27.2) OK, records in: 18919994, records dropped: 87037 output_compression: Snappy 2024/11/23-17:53:36.875627 139957492123200 (Original Log Time 2024/11/23-17:53:36.875588) EVENT_LOG_v1 {"time_micros": 1732355616874706, "cf": "default", "job": 32417, "event": "compaction_finished", "compaction_time_micros": 66886779, "compaction_time_cpu_micros": 28084704, "output_level": 3, "num_output_files": 21, "total_output_size": 772741164, "num_input_records": 18919994, "num_output_records": 18832957, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 526257193, "file_range_sync_nanos": 0, "file_fsync_nanos": 15569, "file_prepare_write_nanos": 60049, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/23-17:53:37.009017 139957492123200 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732355549,"executesMs":67131,"compactionJobId":32417,"attempt":0,"compactionInputRawBytes":2734548224,"compactionInputZipBytes":797875718,"compactionOutputRawBytes":0,"compactionOutputZipBytes":772741164,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-31b6dd74cd82701c847616d6171bd083-07cc2a20d124ec11-00"}}, response = 2024/11/23-17:53:37.009082 139957492123200 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32417/att-00: olev 3, work 67.131 s, result 7.775 ms, install 2.453 ms, input{raw 2.547 GiB zip 760.914 MiB}