2024/11/23-18:17:49.722203 139957618013760 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00: fork to child time = 0.020198 sec 2024/11/23-18:17:49.723093 139957618013760 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00 2024/11/23-18:17:49.731132 139957618013760 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00 2024/11/23-18:17:49.732335 139957618013760 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-18:17:49.732661 139957618013760 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-18:17:49.786398 139957618013760 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32918/att-00/MANIFEST-206347 2024/11/23-18:17:49.801897 139957618013760 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32918/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 274413, last_sequence is 4177273467, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 274240 2024/11/23-18:17:49.801907 139957618013760 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-18:17:49.802083 139957618013760 [version_set.cc:5721] Creating manifest 274413 2024/11/23-18:17:49.838234 139957618013760 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-18:17:49.859232 139957618013760 [compaction_job.cc:2437] [default] [JOB 32918] Compacting 1@2 + 19@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-18:17:49.859254 139957618013760 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24642 Base level 2, inputs: [274380(27MB)], [274405(38MB) 274406(4058KB) 274329(37MB) 274331(35MB) 274332(38MB) 274333(36MB) 274334(35MB) 274335(34MB) 274336(36MB) 274337(34MB) 274338(37MB) 274339(33MB) 274340(34MB) 274341(36MB) 274342(37MB) 274343(37MB) 274344(37MB) 274345(35MB) 274407(32MB)] 2024/11/23-18:17:49.867156 139957618013760 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:17:49.867195 139957618013760 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:17:49.867221 139957618013760 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:17:49.867230 139957618013760 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:17:49.867246 139957618013760 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:17:50.554998 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274414.sst) /dev/shm/ToplingTemp/Topling-147148-Pb5W3Q 2024/11/23-18:17:51.113893 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274414.sst): first pass time = 0.56's, 240.171'MB/sec 2024/11/23-18:17:52.716571 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274414.sst): old prealloc_size = 147639500, real_size = 36172192 2024/11/23-18:17:52.717134 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274414.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930268 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0342 GB avg-val = 116.28 avg-zval = 36.73 2024/11/23-18:17:52.717998 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274414.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:17:53.779145 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274414.sst): second pass time = 1.60's, 67.546'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 38.131'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3225.223'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.263'MB/sec write SST data time = 0.18's, 230.074'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35092.193'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.546'MB/sec zip pipeline throughput = 67.546'MB/sec entries = 930268 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.28 avg-zval = 36.73 usrkeys = 930268 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.28 avg-zval = 36.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930268 bytes = 3488512 } seq expand size = 6511876 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.5135 value = 3.1657 dict = 1.00 all = 3.3127 } Zip/UnZip{ index = 0.0951 value = 0.3159 dict = 1.00 all = 0.3019 } ---------------------------- total value len = 8.963302 GB avg = 0.168 KB (by entry num) total key len = 1.466869 GB avg = 0.028 KB total ukey len = 1.016483 GB avg = 0.015 KB total ukey num = 0.066480703 Billion total entry num = 0.053288432 Billion write speed all = 0.308936235 MB/sec (with seq num) write speed all = 0.296309229 MB/sec (without seq num) 2024/11/23-18:17:53.832982 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274414: 930268 keys, 40516510 bytes, temperature: kWarm 2024/11/23-18:17:53.833076 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357073833019, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274414, "file_size": 40516510, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4177020028, "table_properties": {"data_size": 34169808, "index_size": 1769664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26047504, "raw_average_key_size": 28, "raw_value_size": 108170356, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930268, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357070, "oldest_key_time": 0, "file_creation_time": 1732357070, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274414, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:17:53.870109 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274415.sst) /dev/shm/ToplingTemp/Topling-147148-xe47MO 2024/11/23-18:17:54.896208 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274415.sst): first pass time = 1.03's, 130.808'MB/sec 2024/11/23-18:17:56.598144 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274415.sst): old prealloc_size = 147639500, real_size = 36177328 2024/11/23-18:17:56.598826 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274415.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934275 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0341 GB avg-val = 115.66 avg-zval = 36.54 2024/11/23-18:17:56.599544 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274415.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:17:57.555629 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274415.sst): second pass time = 1.70's, 63.548'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 41.287'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2727.333'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.296'MB/sec write SST data time = 0.17's, 235.869'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34339.172'MB/sec zip my value time = 1.70's, unzip length = 0.108'GB zip my value throughput = 63.548'MB/sec zip pipeline throughput = 63.548'MB/sec entries = 934275 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.66 avg-zval = 36.54 usrkeys = 934275 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.36 avg-zval = 36.54 TagRS{ kinds = RS_Key0_TagN bytes = 146080 } TagArray{ size = 627483 bytes = 2353072 } seq expand size = 4392381 multi value expand size = 0 cnt WriteAppend = 20 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.3764 value = 3.1649 dict = 1.00 all = 3.3953 } Zip/UnZip{ index = 0.0964 value = 0.3160 dict = 1.00 all = 0.2945 } ---------------------------- total value len = 9.075752 GB avg = 0.167 KB (by entry num) total key len = 1.493028 GB avg = 0.028 KB total ukey len = 1.035169 GB avg = 0.015 KB total ukey num = 0.067414978 Billion total entry num = 0.054222707 Billion write speed all = 0.313006782 MB/sec (with seq num) write speed all = 0.300159832 MB/sec (without seq num) 2024/11/23-18:17:57.607489 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274415: 934275 keys, 39530746 bytes, temperature: kWarm 2024/11/23-18:17:57.607560 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357077607523, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274415, "file_size": 39530746, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34142816, "index_size": 1800768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26159700, "raw_average_key_size": 28, "raw_value_size": 108058038, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934275, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357073, "oldest_key_time": 0, "file_creation_time": 1732357073, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274415, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:17:57.645879 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274416.sst) /dev/shm/ToplingTemp/Topling-147148-z7QvsO 2024/11/23-18:17:58.678572 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274416.sst): first pass time = 1.03's, 129.974'MB/sec 2024/11/23-18:18:00.080007 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274416.sst): old prealloc_size = 147639500, real_size = 33148432 2024/11/23-18:18:00.080689 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274416.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924389 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.20 avg-zval = 33.71 2024/11/23-18:18:00.081559 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274416.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:01.105040 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274416.sst): second pass time = 1.40's, 77.350'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 36.444'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2668.873'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.265'MB/sec write SST data time = 0.15's, 246.803'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34732.077'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.350'MB/sec zip pipeline throughput = 77.350'MB/sec entries = 924389 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.20 avg-zval = 33.71 usrkeys = 924389 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.20 avg-zval = 33.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924389 bytes = 3350928 } seq expand size = 6470723 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5151 value = 3.4769 dict = 1.00 all = 3.5930 } Zip/UnZip{ index = 0.0951 value = 0.2876 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 9.190558 GB avg = 0.167 KB (by entry num) total key len = 1.518911 GB avg = 0.028 KB total ukey len = 1.053657 GB avg = 0.015 KB total ukey num = 0.068339367 Billion total entry num = 0.055147096 Billion write speed all = 0.317140099 MB/sec (with seq num) write speed all = 0.304075507 MB/sec (without seq num) 2024/11/23-18:18:01.153909 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274416: 924389 keys, 37355864 bytes, temperature: kWarm 2024/11/23-18:18:01.153996 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357081153945, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274416, "file_size": 37355864, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3624371836, "largest_seqno": 4012775935, "table_properties": {"data_size": 31158912, "index_size": 1758208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25882892, "raw_average_key_size": 28, "raw_value_size": 108334878, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924389, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357077, "oldest_key_time": 0, "file_creation_time": 1732357077, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274416, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:01.190936 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274417.sst) /dev/shm/ToplingTemp/Topling-147148-IMYTuP 2024/11/23-18:18:02.287833 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274417.sst): first pass time = 1.10's, 122.367'MB/sec 2024/11/23-18:18:03.783903 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274417.sst): old prealloc_size = 147639500, real_size = 35990496 2024/11/23-18:18:03.784640 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274417.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925167 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0340 GB avg-val = 117.07 avg-zval = 36.74 2024/11/23-18:18:03.785554 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274417.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:04.730824 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274417.sst): second pass time = 1.50's, 72.445'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 42.454'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2475.011'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.15's, 261.975'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32656.597'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.445'MB/sec zip pipeline throughput = 72.445'MB/sec entries = 925167 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.07 avg-zval = 36.74 usrkeys = 925167 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.07 avg-zval = 36.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925167 bytes = 3353744 } seq expand size = 6476169 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.4434 value = 3.1869 dict = 1.00 all = 3.3388 } Zip/UnZip{ index = 0.0958 value = 0.3138 dict = 1.00 all = 0.2995 } ---------------------------- total value len = 9.305347 GB avg = 0.166 KB (by entry num) total key len = 1.544816 GB avg = 0.028 KB total ukey len = 1.072160 GB avg = 0.015 KB total ukey num = 0.069264534 Billion total entry num = 0.056072263 Billion write speed all = 0.321271980 MB/sec (with seq num) write speed all = 0.307989639 MB/sec (without seq num) 2024/11/23-18:18:04.783228 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274417: 925167 keys, 40199800 bytes, temperature: kWarm 2024/11/23-18:18:04.783295 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357084783259, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274417, "file_size": 40199800, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33987216, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904676, "raw_average_key_size": 28, "raw_value_size": 108313086, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925167, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357081, "oldest_key_time": 0, "file_creation_time": 1732357081, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274417, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:04.821357 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274418.sst) /dev/shm/ToplingTemp/Topling-147148-PKfRxS 2024/11/23-18:18:05.901607 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274418.sst): first pass time = 1.08's, 124.251'MB/sec 2024/11/23-18:18:07.380222 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274418.sst): old prealloc_size = 147639500, real_size = 33370224 2024/11/23-18:18:07.381065 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274418.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929618 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.38 avg-zval = 33.74 2024/11/23-18:18:07.381960 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274418.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:08.349891 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274418.sst): second pass time = 1.48's, 73.246'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.828'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2143.052'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.288'MB/sec write SST data time = 0.16's, 234.165'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34704.479'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.246'MB/sec zip pipeline throughput = 73.246'MB/sec entries = 929618 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.38 avg-zval = 33.74 usrkeys = 929618 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.96 avg-zval = 33.74 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 874162 bytes = 3278128 } seq expand size = 6119134 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5141 value = 3.4489 dict = 1.00 all = 3.5649 } Zip/UnZip{ index = 0.0951 value = 0.2900 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 9.419655 GB avg = 0.165 KB (by entry num) total key len = 1.570845 GB avg = 0.028 KB total ukey len = 1.090752 GB avg = 0.016 KB total ukey num = 0.070194152 Billion total entry num = 0.057001881 Billion write speed all = 0.325392471 MB/sec (with seq num) write speed all = 0.311891369 MB/sec (without seq num) 2024/11/23-18:18:08.397574 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274418: 929618 keys, 37650240 bytes, temperature: kWarm 2024/11/23-18:18:08.397644 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357088397607, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274418, "file_size": 37650240, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 31369312, "index_size": 1768320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26029304, "raw_average_key_size": 28, "raw_value_size": 108188528, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929618, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357084, "oldest_key_time": 0, "file_creation_time": 1732357084, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274418, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:08.434444 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274419.sst) /dev/shm/ToplingTemp/Topling-147148-gFm69R 2024/11/23-18:18:09.577014 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274419.sst): first pass time = 1.14's, 117.474'MB/sec 2024/11/23-18:18:10.903067 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274419.sst): old prealloc_size = 147639500, real_size = 32616256 2024/11/23-18:18:10.903665 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274419.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930032 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.32 avg-zval = 32.92 2024/11/23-18:18:10.904436 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274419.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:11.730074 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274419.sst): second pass time = 1.33's, 81.628'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.290'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3077.354'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 281.941'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35904.816'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.628'MB/sec zip pipeline throughput = 81.628'MB/sec entries = 930032 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.32 avg-zval = 32.92 usrkeys = 930032 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.36 avg-zval = 32.92 TagRS{ kinds = RS_Key0_TagN bytes = 145400 } TagArray{ size = 802491 bytes = 3009360 } seq expand size = 5617437 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5143 value = 3.5335 dict = 1.00 all = 3.6645 } Zip/UnZip{ index = 0.0951 value = 0.2830 dict = 1.00 all = 0.2729 } ---------------------------- total value len = 9.533449 GB avg = 0.165 KB (by entry num) total key len = 1.596886 GB avg = 0.028 KB total ukey len = 1.109353 GB avg = 0.016 KB total ukey num = 0.071124184 Billion total entry num = 0.057931913 Billion write speed all = 0.329499558 MB/sec (with seq num) write speed all = 0.315779548 MB/sec (without seq num) 2024/11/23-18:18:11.778802 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274419: 930032 keys, 36626944 bytes, temperature: kWarm 2024/11/23-18:18:11.778871 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357091778835, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274419, "file_size": 36626944, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 30614448, "index_size": 1769088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26040896, "raw_average_key_size": 28, "raw_value_size": 108176956, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930032, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357088, "oldest_key_time": 0, "file_creation_time": 1732357088, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274419, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:11.820757 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274420.sst) /dev/shm/ToplingTemp/Topling-147148-r2K11R 2024/11/23-18:18:12.768695 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274420.sst): first pass time = 0.95's, 141.594'MB/sec 2024/11/23-18:18:14.253922 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274420.sst): old prealloc_size = 147639500, real_size = 31484192 2024/11/23-18:18:14.254633 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274420.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931692 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0295 GB avg-val = 116.06 avg-zval = 31.64 2024/11/23-18:18:14.255521 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274420.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:15.148199 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274420.sst): second pass time = 1.48's, 72.843'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 39.935'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2574.903'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.15's, 230.416'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37331.348'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 72.843'MB/sec zip pipeline throughput = 72.843'MB/sec entries = 931692 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 31.64 usrkeys = 931692 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 31.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931692 bytes = 3377392 } seq expand size = 6521844 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5140 value = 3.6681 dict = 1.00 all = 3.7583 } Zip/UnZip{ index = 0.0951 value = 0.2726 dict = 1.00 all = 0.2661 } ---------------------------- total value len = 9.648102 GB avg = 0.164 KB (by entry num) total key len = 1.622973 GB avg = 0.028 KB total ukey len = 1.127987 GB avg = 0.016 KB total ukey num = 0.072055876 Billion total entry num = 0.058863605 Billion write speed all = 0.333632218 MB/sec (with seq num) write speed all = 0.319692965 MB/sec (without seq num) 2024/11/23-18:18:15.193028 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274420: 931692 keys, 35712540 bytes, temperature: kWarm 2024/11/23-18:18:15.193096 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357095193061, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274420, "file_size": 35712540, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776228, "table_properties": {"data_size": 29478800, "index_size": 1772288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26087376, "raw_average_key_size": 28, "raw_value_size": 108130408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931692, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357091, "oldest_key_time": 0, "file_creation_time": 1732357091, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274420, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:15.231137 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274421.sst) /dev/shm/ToplingTemp/Topling-147148-PHsh2P 2024/11/23-18:18:16.365636 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274421.sst): first pass time = 1.13's, 118.309'MB/sec 2024/11/23-18:18:17.693072 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274421.sst): old prealloc_size = 147639500, real_size = 32945424 2024/11/23-18:18:17.693626 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274421.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933210 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 115.82 avg-zval = 33.14 2024/11/23-18:18:17.694417 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274421.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:18.548215 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274421.sst): second pass time = 1.33's, 81.474'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.478'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3380.841'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 294.682'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32782.754'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.474'MB/sec zip pipeline throughput = 81.474'MB/sec entries = 933210 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.82 avg-zval = 33.14 usrkeys = 933210 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.82 avg-zval = 33.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933210 bytes = 3382896 } seq expand size = 6532470 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4444 value = 3.4952 dict = 1.00 all = 3.6100 } Zip/UnZip{ index = 0.0957 value = 0.2861 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 9.762722 GB avg = 0.163 KB (by entry num) total key len = 1.649103 GB avg = 0.028 KB total ukey len = 1.146651 GB avg = 0.016 KB total ukey num = 0.072989086 Billion total entry num = 0.059796815 Billion write speed all = 0.337764535 MB/sec (with seq num) write speed all = 0.323605717 MB/sec (without seq num) 2024/11/23-18:18:18.594608 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274421: 933210 keys, 37179072 bytes, temperature: kWarm 2024/11/23-18:18:18.594699 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357098594643, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274421, "file_size": 37179072, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 4012776268, "table_properties": {"data_size": 30924928, "index_size": 1787008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26129880, "raw_average_key_size": 28, "raw_value_size": 108087848, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933210, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357095, "oldest_key_time": 0, "file_creation_time": 1732357095, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274421, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:18.639950 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274422.sst) /dev/shm/ToplingTemp/Topling-147148-UftDtO 2024/11/23-18:18:19.512284 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274422.sst): first pass time = 0.87's, 153.871'MB/sec 2024/11/23-18:18:20.829244 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274422.sst): old prealloc_size = 147639500, real_size = 33146560 2024/11/23-18:18:20.829782 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274422.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923648 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0311 GB avg-val = 117.31 avg-zval = 33.71 2024/11/23-18:18:20.830598 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274422.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:21.667063 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274422.sst): second pass time = 1.32's, 82.325'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.722'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3421.868'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 297.401'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35540.482'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.325'MB/sec zip pipeline throughput = 82.325'MB/sec entries = 923648 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.31 avg-zval = 33.71 usrkeys = 923648 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.31 avg-zval = 33.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923648 bytes = 3463696 } seq expand size = 6465536 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.3753 value = 3.4802 dict = 1.00 all = 3.5822 } Zip/UnZip{ index = 0.0964 value = 0.2873 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 9.877543 GB avg = 0.163 KB (by entry num) total key len = 1.674965 GB avg = 0.028 KB total ukey len = 1.165124 GB avg = 0.016 KB total ukey num = 0.073912734 Billion total entry num = 0.060720463 Billion write speed all = 0.341896888 MB/sec (with seq num) write speed all = 0.327520694 MB/sec (without seq num) 2024/11/23-18:18:21.701765 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274422: 923648 keys, 37467660 bytes, temperature: kWarm 2024/11/23-18:18:21.701836 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357101701799, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274422, "file_size": 37467660, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 31134960, "index_size": 1780480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25862144, "raw_average_key_size": 28, "raw_value_size": 108355700, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923648, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357098, "oldest_key_time": 0, "file_creation_time": 1732357098, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274422, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:21.738148 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274423.sst) /dev/shm/ToplingTemp/Topling-147148-gQ4JwR 2024/11/23-18:18:22.692581 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274423.sst): first pass time = 0.95's, 140.631'MB/sec 2024/11/23-18:18:24.009374 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274423.sst): old prealloc_size = 147639500, real_size = 34349280 2024/11/23-18:18:24.009900 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274423.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930375 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.26 avg-zval = 34.76 2024/11/23-18:18:24.010704 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274423.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:24.854238 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274423.sst): second pass time = 1.32's, 82.196'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.657'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3536.419'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 302.727'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29647.366'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.196'MB/sec zip pipeline throughput = 82.196'MB/sec entries = 930375 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.26 avg-zval = 34.76 usrkeys = 930375 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.26 avg-zval = 34.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930375 bytes = 3372624 } seq expand size = 6512625 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.4460 value = 3.3452 dict = 1.00 all = 3.4795 } Zip/UnZip{ index = 0.0957 value = 0.2989 dict = 1.00 all = 0.2874 } ---------------------------- total value len = 9.992223 GB avg = 0.162 KB (by entry num) total key len = 1.701016 GB avg = 0.028 KB total ukey len = 1.183731 GB avg = 0.016 KB total ukey num = 0.074843109 Billion total entry num = 0.061650838 Billion write speed all = 0.346029168 MB/sec (with seq num) write speed all = 0.331434076 MB/sec (without seq num) 2024/11/23-18:18:24.902517 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274423: 930375 keys, 38573454 bytes, temperature: kWarm 2024/11/23-18:18:24.902590 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357104902553, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274423, "file_size": 38573454, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 32335184, "index_size": 1781312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26050500, "raw_average_key_size": 28, "raw_value_size": 108167242, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930375, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357101, "oldest_key_time": 0, "file_creation_time": 1732357101, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274423, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:24.937962 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274424.sst) /dev/shm/ToplingTemp/Topling-147148-rSRBRO 2024/11/23-18:18:25.932716 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274424.sst): first pass time = 0.99's, 134.933'MB/sec 2024/11/23-18:18:27.223042 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274424.sst): old prealloc_size = 147639500, real_size = 31712224 2024/11/23-18:18:27.223586 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274424.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928678 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0297 GB avg-val = 116.53 avg-zval = 31.98 2024/11/23-18:18:27.224423 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274424.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:28.052281 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274424.sst): second pass time = 1.29's, 83.924'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.470'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3405.156'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 287.463'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34009.512'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.924'MB/sec zip pipeline throughput = 83.924'MB/sec entries = 928678 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.53 avg-zval = 31.98 usrkeys = 928678 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.53 avg-zval = 31.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928678 bytes = 3482560 } seq expand size = 6500746 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4434 value = 3.6434 dict = 1.00 all = 3.7233 } Zip/UnZip{ index = 0.0958 value = 0.2745 dict = 1.00 all = 0.2686 } ---------------------------- total value len = 10.106939 GB avg = 0.162 KB (by entry num) total key len = 1.727019 GB avg = 0.028 KB total ukey len = 1.202305 GB avg = 0.016 KB total ukey num = 0.075771787 Billion total entry num = 0.062579516 Billion write speed all = 0.350160207 MB/sec (with seq num) write speed all = 0.335346663 MB/sec (without seq num) 2024/11/23-18:18:28.097383 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274424: 928678 keys, 36048232 bytes, temperature: kWarm 2024/11/23-18:18:28.097463 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357108097418, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274424, "file_size": 36048232, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 29701392, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002984, "raw_average_key_size": 28, "raw_value_size": 108214792, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928678, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357104, "oldest_key_time": 0, "file_creation_time": 1732357104, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274424, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:28.134956 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274425.sst) /dev/shm/ToplingTemp/Topling-147148-znDbiS 2024/11/23-18:18:29.140945 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274425.sst): first pass time = 1.01's, 133.428'MB/sec 2024/11/23-18:18:30.467179 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274425.sst): old prealloc_size = 147639500, real_size = 32545248 2024/11/23-18:18:30.467717 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274425.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923122 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.40 avg-zval = 33.08 2024/11/23-18:18:30.468535 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274425.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:31.294029 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274425.sst): second pass time = 1.32's, 81.794'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.587'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3445.490'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.12's, 296.089'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31397.738'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.794'MB/sec zip pipeline throughput = 81.794'MB/sec entries = 923122 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.40 avg-zval = 33.08 usrkeys = 923122 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.40 avg-zval = 33.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923122 bytes = 3461728 } seq expand size = 6461854 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.3750 value = 3.5491 dict = 1.00 all = 3.6408 } Zip/UnZip{ index = 0.0964 value = 0.2818 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 10.221771 GB avg = 0.161 KB (by entry num) total key len = 1.752866 GB avg = 0.028 KB total ukey len = 1.220767 GB avg = 0.016 KB total ukey num = 0.076694909 Billion total entry num = 0.063502638 Billion write speed all = 0.354288857 MB/sec (with seq num) write speed all = 0.339258238 MB/sec (without seq num) 2024/11/23-18:18:31.339418 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274425: 923122 keys, 36864562 bytes, temperature: kWarm 2024/11/23-18:18:31.339491 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357111339453, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274425, "file_size": 36864562, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776360, "table_properties": {"data_size": 30534736, "index_size": 1779520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847416, "raw_average_key_size": 28, "raw_value_size": 108370436, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923122, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357108, "oldest_key_time": 0, "file_creation_time": 1732357108, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274425, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:31.377758 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274426.sst) /dev/shm/ToplingTemp/Topling-147148-J6cN2R 2024/11/23-18:18:32.179821 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274426.sst): first pass time = 0.80's, 167.348'MB/sec 2024/11/23-18:18:33.646335 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274426.sst): old prealloc_size = 147639500, real_size = 36171008 2024/11/23-18:18:33.646852 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274426.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930566 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0342 GB avg-val = 116.23 avg-zval = 36.72 2024/11/23-18:18:33.647134 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274426.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:34.369118 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274426.sst): second pass time = 1.47's, 73.798'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 52.238'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3565.372'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.389'MB/sec write SST data time = 0.12's, 305.266'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35647.168'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.798'MB/sec zip pipeline throughput = 73.798'MB/sec entries = 930566 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.23 avg-zval = 36.72 usrkeys = 930566 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.64 avg-zval = 36.72 TagRS{ kinds = RS_Key0_TagN bytes = 145520 } TagArray{ size = 186980 bytes = 584320 } seq expand size = 1308860 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5123 value = 3.1656 dict = 1.00 all = 3.5548 } Zip/UnZip{ index = 0.0951 value = 0.3159 dict = 1.00 all = 0.2813 } ---------------------------- total value len = 10.331242 GB avg = 0.160 KB (by entry num) total key len = 1.778922 GB avg = 0.028 KB total ukey len = 1.239379 GB avg = 0.016 KB total ukey num = 0.077625475 Billion total entry num = 0.064433204 Billion write speed all = 0.358266034 MB/sec (with seq num) write speed all = 0.343016544 MB/sec (without seq num) 2024/11/23-18:18:34.417117 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274426: 930566 keys, 37756677 bytes, temperature: kWarm 2024/11/23-18:18:34.417184 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357114417150, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274426, "file_size": 37756677, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012773626, "table_properties": {"data_size": 34167728, "index_size": 1770432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26055848, "raw_average_key_size": 28, "raw_value_size": 108161916, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930566, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357111, "oldest_key_time": 0, "file_creation_time": 1732357111, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274426, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:34.454042 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274427.sst) /dev/shm/ToplingTemp/Topling-147148-iWxzlS 2024/11/23-18:18:35.427650 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274427.sst): first pass time = 0.97's, 137.861'MB/sec 2024/11/23-18:18:36.894786 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274427.sst): old prealloc_size = 147639500, real_size = 37269840 2024/11/23-18:18:36.895353 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274427.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924974 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0353 GB avg-val = 117.10 avg-zval = 38.13 2024/11/23-18:18:36.895784 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274427.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:37.635361 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274427.sst): second pass time = 1.47's, 73.869'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 53.541'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3230.723'MB/sec (index lex order gen) rebuild zvType time = 0.62's, 0.376'MB/sec write SST data time = 0.12's, 317.249'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33878.315'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.869'MB/sec zip pipeline throughput = 73.869'MB/sec entries = 924974 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.10 avg-zval = 38.13 usrkeys = 924974 avg-key = 20.00 avg-zkey = 1.91 avg-val = 119.88 avg-zval = 38.13 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 366967 bytes = 1376144 } seq expand size = 2568769 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0353 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4457 value = 3.0713 dict = 1.00 all = 3.3851 } Zip/UnZip{ index = 0.0957 value = 0.3256 dict = 1.00 all = 0.2954 } ---------------------------- total value len = 10.442129 GB avg = 0.160 KB (by entry num) total key len = 1.804821 GB avg = 0.028 KB total ukey len = 1.257878 GB avg = 0.016 KB total ukey num = 0.078550449 Billion total entry num = 0.065358178 Billion write speed all = 0.362277708 MB/sec (with seq num) write speed all = 0.346810798 MB/sec (without seq num) 2024/11/23-18:18:37.684039 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274427: 924974 keys, 39649697 bytes, temperature: kWarm 2024/11/23-18:18:37.684110 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357117684073, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274427, "file_size": 39649697, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4168991613, "table_properties": {"data_size": 35267392, "index_size": 1771008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25899272, "raw_average_key_size": 28, "raw_value_size": 108318468, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924974, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357114, "oldest_key_time": 0, "file_creation_time": 1732357114, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274427, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:37.726350 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274428.sst) /dev/shm/ToplingTemp/Topling-147148-uwQqfO 2024/11/23-18:18:38.636006 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274428.sst): first pass time = 0.91's, 147.554'MB/sec 2024/11/23-18:18:39.990921 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274428.sst): old prealloc_size = 147639500, real_size = 35286560 2024/11/23-18:18:39.991500 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274428.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928434 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0333 GB avg-val = 116.56 avg-zval = 35.85 2024/11/23-18:18:39.992236 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274428.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:40.823940 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274428.sst): second pass time = 1.35's, 79.950'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.147'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3171.555'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 312.532'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32667.857'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.950'MB/sec zip pipeline throughput = 79.950'MB/sec entries = 928434 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 35.85 usrkeys = 928434 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 35.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928434 bytes = 3133472 } seq expand size = 6499038 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5137 value = 3.2511 dict = 1.00 all = 3.4176 } Zip/UnZip{ index = 0.0951 value = 0.3076 dict = 1.00 all = 0.2926 } ---------------------------- total value len = 10.556850 GB avg = 0.159 KB (by entry num) total key len = 1.830818 GB avg = 0.028 KB total ukey len = 1.276447 GB avg = 0.016 KB total ukey num = 0.079478883 Billion total entry num = 0.066286612 Billion write speed all = 0.366405703 MB/sec (with seq num) write speed all = 0.350720559 MB/sec (without seq num) 2024/11/23-18:18:40.873142 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274428: 928434 keys, 39273090 bytes, temperature: kWarm 2024/11/23-18:18:40.873210 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357120873175, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274428, "file_size": 39273090, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4177019454, "table_properties": {"data_size": 33288144, "index_size": 1766144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25996152, "raw_average_key_size": 28, "raw_value_size": 108221720, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357117, "oldest_key_time": 0, "file_creation_time": 1732357117, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274428, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:40.910894 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274429.sst) /dev/shm/ToplingTemp/Topling-147148-I7sLoO 2024/11/23-18:18:41.872836 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274429.sst): first pass time = 0.96's, 139.533'MB/sec 2024/11/23-18:18:43.229022 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274429.sst): old prealloc_size = 147639500, real_size = 35228544 2024/11/23-18:18:43.229572 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274429.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927258 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 116.75 avg-zval = 35.84 2024/11/23-18:18:43.230309 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274429.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:44.061697 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274429.sst): second pass time = 1.35's, 79.899'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 46.953'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3348.892'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 310.025'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 25846.485'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.899'MB/sec zip pipeline throughput = 79.899'MB/sec entries = 927258 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 35.84 usrkeys = 927258 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 35.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927258 bytes = 3013600 } seq expand size = 6490806 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5129 value = 3.2575 dict = 1.00 all = 3.4330 } Zip/UnZip{ index = 0.0951 value = 0.3070 dict = 1.00 all = 0.2913 } ---------------------------- total value len = 10.671595 GB avg = 0.159 KB (by entry num) total key len = 1.856781 GB avg = 0.028 KB total ukey len = 1.294992 GB avg = 0.016 KB total ukey num = 0.080406141 Billion total entry num = 0.067213870 Billion write speed all = 0.370532135 MB/sec (with seq num) write speed all = 0.354629100 MB/sec (without seq num) 2024/11/23-18:18:44.109539 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274429: 927258 keys, 39096708 bytes, temperature: kWarm 2024/11/23-18:18:44.109608 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357124109573, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274429, "file_size": 39096708, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4127562387, "largest_seqno": 4177019963, "table_properties": {"data_size": 33232496, "index_size": 1764032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963224, "raw_average_key_size": 28, "raw_value_size": 108254580, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927258, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357120, "oldest_key_time": 0, "file_creation_time": 1732357120, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274429, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:44.151956 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274430.sst) /dev/shm/ToplingTemp/Topling-147148-tf3aMR 2024/11/23-18:18:45.169885 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274430.sst): first pass time = 1.02's, 131.861'MB/sec 2024/11/23-18:18:46.495767 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274430.sst): old prealloc_size = 147639500, real_size = 34981184 2024/11/23-18:18:46.496333 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274430.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930817 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0330 GB avg-val = 116.19 avg-zval = 35.42 2024/11/23-18:18:46.497099 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274430.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:47.327107 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274430.sst): second pass time = 1.33's, 81.621'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 46.887'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3281.710'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 308.878'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34844.544'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.621'MB/sec zip pipeline throughput = 81.621'MB/sec entries = 930817 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.19 avg-zval = 35.42 usrkeys = 930817 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.19 avg-zval = 35.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930817 bytes = 3141520 } seq expand size = 6515719 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4457 value = 3.2808 dict = 1.00 all = 3.4434 } Zip/UnZip{ index = 0.0957 value = 0.3048 dict = 1.00 all = 0.2904 } ---------------------------- total value len = 10.786266 GB avg = 0.158 KB (by entry num) total key len = 1.882844 GB avg = 0.028 KB total ukey len = 1.313608 GB avg = 0.016 KB total ukey num = 0.081336958 Billion total entry num = 0.068144687 Billion write speed all = 0.374658205 MB/sec (with seq num) write speed all = 0.358536493 MB/sec (without seq num) 2024/11/23-18:18:47.375429 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274430: 930817 keys, 38978178 bytes, temperature: kWarm 2024/11/23-18:18:47.375500 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357127375464, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274430, "file_size": 38978178, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4096159344, "largest_seqno": 4177020194, "table_properties": {"data_size": 32966064, "index_size": 1782208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26062876, "raw_average_key_size": 28, "raw_value_size": 108154866, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930817, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357124, "oldest_key_time": 0, "file_creation_time": 1732357124, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274430, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:47.415934 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274431.sst) /dev/shm/ToplingTemp/Topling-147148-w2SB7P 2024/11/23-18:18:48.399588 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274431.sst): first pass time = 0.98's, 136.453'MB/sec 2024/11/23-18:18:49.721632 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274431.sst): old prealloc_size = 147639500, real_size = 32736448 2024/11/23-18:18:49.722195 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274431.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927530 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.70 avg-zval = 33.14 2024/11/23-18:18:49.724152 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274431.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:18:50.569521 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274431.sst): second pass time = 1.32's, 81.929'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.310'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3268.298'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 286.580'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32583.572'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.929'MB/sec zip pipeline throughput = 81.929'MB/sec entries = 927530 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.70 avg-zval = 33.14 usrkeys = 927530 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.70 avg-zval = 33.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927530 bytes = 3130432 } seq expand size = 6492710 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5149 value = 3.5214 dict = 1.00 all = 3.6551 } Zip/UnZip{ index = 0.0951 value = 0.2840 dict = 1.00 all = 0.2736 } ---------------------------- total value len = 10.901005 GB avg = 0.158 KB (by entry num) total key len = 1.908815 GB avg = 0.028 KB total ukey len = 1.332159 GB avg = 0.016 KB total ukey num = 0.082264488 Billion total entry num = 0.069072217 Billion write speed all = 0.378783058 MB/sec (with seq num) write speed all = 0.362443477 MB/sec (without seq num) 2024/11/23-18:18:50.614987 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274431: 927530 keys, 36721026 bytes, temperature: kWarm 2024/11/23-18:18:50.615059 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357130615022, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274431, "file_size": 36721026, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4076718630, "largest_seqno": 4177020206, "table_properties": {"data_size": 30740144, "index_size": 1764224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25970840, "raw_average_key_size": 28, "raw_value_size": 108247000, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927530, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357127, "oldest_key_time": 0, "file_creation_time": 1732357127, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274431, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:50.655422 139957618013760 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274432.sst) /dev/shm/ToplingTemp/Topling-147148-mjGweS 2024/11/23-18:18:50.805950 139957618013760 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274432.sst): first pass time = 0.15's, 242.835'MB/sec 2024/11/23-18:18:51.147570 139957618013760 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274432.sst): old prealloc_size = 147639500, real_size = 9260896 2024/11/23-18:18:51.147771 139957618013760 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274432.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 257351 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0293 GB zip-val = 0.0087 GB avg-val = 114.00 avg-zval = 33.82 2024/11/23-18:18:51.148038 139957618013760 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274432.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.037 GB 2024/11/23-18:18:51.377937 139957618013760 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00/274432.sst): second pass time = 0.34's, 86.065'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.23's, 44.723'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2779.117'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.332'MB/sec write SST data time = 0.04's, 282.674'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.299'MB, throughput = 11116.270'MB/sec zip my value time = 0.34's, unzip length = 0.029'GB zip my value throughput = 86.065'MB/sec zip pipeline throughput = 86.065'MB/sec entries = 257351 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.00 avg-zval = 33.82 usrkeys = 257351 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.00 avg-zval = 33.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 257351 bytes = 804240 } seq expand size = 1801457 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0051 GB value = 0.0293 GB dict = 0.30 MB all = 0.0365 GB } __ZipSize{ index = 0.0005 GB value = 0.0087 GB dict = 0.30 MB all = 0.0103 GB } UnZip/Zip{ index = 10.4214 value = 3.3712 dict = 1.00 all = 3.5473 } Zip/UnZip{ index = 0.0960 value = 0.2966 dict = 1.00 all = 0.2819 } ---------------------------- total value len = 10.932145 GB avg = 0.158 KB (by entry num) total key len = 1.916020 GB avg = 0.028 KB total ukey len = 1.337306 GB avg = 0.016 KB total ukey num = 0.082521839 Billion total entry num = 0.069329568 Billion write speed all = 0.379907837 MB/sec (with seq num) write speed all = 0.363507770 MB/sec (without seq num) 2024/11/23-18:18:51.395665 139957618013760 [compaction_job.cc:1948] [default] [JOB 32918] Generated table #274432: 257351 keys, 10301777 bytes, temperature: kWarm 2024/11/23-18:18:51.395731 139957618013760 EVENT_LOG_v1 {"time_micros": 1732357131395697, "cf_name": "default", "job": 32918, "event": "table_file_creation", "file_number": 274432, "file_size": 10301777, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4151669401, "largest_seqno": 4176429128, "table_properties": {"data_size": 8702480, "index_size": 493888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7205828, "raw_average_key_size": 28, "raw_value_size": 29338014, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 257351, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732357130, "oldest_key_time": 0, "file_creation_time": 1732357130, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 274432, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:18:51.410085 139957618013760 [compaction_job.cc:703] job-32918: subcompact[0], size: 0.695503 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32918/att-00 [274414,274415,274416,274417,274418,274419,274420,274421,274422,274423,274424,274425,274426,274427,274428,274429,274430,274431,274432] 2024/11/23-18:18:51.522744 139957618013760 (Original Log Time 2024/11/23-18:17:49.859305) EVENT_LOG_v1 {"time_micros": 1732357069859267, "job": 32918, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [274380], "files_L3": [274405, 274406, 274329, 274331, 274332, 274333, 274334, 274335, 274336, 274337, 274338, 274339, 274340, 274341, 274342, 274343, 274344, 274345, 274407], "score": 0.986053, "input_data_size": 713510451, "oldest_snapshot_seqno": 4177273425} 2024/11/23-18:18:51.522748 139957618013760 (Original Log Time 2024/11/23-18:18:51.521167) [compaction_job.cc:2022] [default] [JOB 32918] Compacted 1@2 + 19@3 files to L3 => 695502777 bytes 2024/11/23-18:18:51.522750 139957618013760 (Original Log Time 2024/11/23-18:18:51.522669) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 19 0 0 0] max score 0.03, MB/sec: 11.6 rd, 11.3 wr, level 3, files in(1, 19) out(19 +0 blob) MB in(27.1, 653.3 +0.0 blob) out(663.3 +0.0 blob), read-write-amplify(49.5) write-amplify(24.4) OK, records in: 17037385, records dropped: 65981 output_compression: Snappy 2024/11/23-18:18:51.522753 139957618013760 (Original Log Time 2024/11/23-18:18:51.522719) EVENT_LOG_v1 {"time_micros": 1732357131522695, "cf": "default", "job": 32918, "event": "compaction_finished", "compaction_time_micros": 61550788, "compaction_time_cpu_micros": 26560724, "output_level": 3, "num_output_files": 19, "total_output_size": 695502777, "num_input_records": 17037385, "num_output_records": 16971404, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 508106488, "file_range_sync_nanos": 0, "file_fsync_nanos": 12737, "file_prepare_write_nanos": 55704, "lsm_state": [0, 0, 0, 19, 0, 0, 0]} 2024/11/23-18:18:51.640610 139957618013760 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732357069,"executesMs":61792,"compactionJobId":32918,"attempt":0,"compactionInputRawBytes":2461833446,"compactionInputZipBytes":713510451,"compactionOutputRawBytes":0,"compactionOutputZipBytes":695502777,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-0fd9907b54a6aee865f7a7c863d79d53-c2e979bc0b2377ec-00"}}, response = 2024/11/23-18:18:51.640702 139957618013760 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32918/att-00: olev 3, work 61.793 s, result 6.010 ms, install 1.601 ms, input{raw 2.293 GiB zip 680.457 MiB}