2024/11/23-08:58:03.638540 139957508908608 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00: fork to child time = 0.007162 sec 2024/11/23-08:58:03.638988 139957508908608 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00 2024/11/23-08:58:03.645629 139957508908608 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00 2024/11/23-08:58:03.646502 139957508908608 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-08:58:03.646712 139957508908608 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-08:58:03.692932 139957508908608 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-23096/att-00/MANIFEST-206347 2024/11/23-08:58:03.729092 139957508908608 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-23096/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 253281, last_sequence is 3895605423, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 253140 2024/11/23-08:58:03.729102 139957508908608 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-08:58:03.729225 139957508908608 [version_set.cc:5721] Creating manifest 253281 2024/11/23-08:58:03.763231 139957508908608 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-08:58:03.785309 139957508908608 [compaction_job.cc:2437] [default] [JOB 23096] Compacting 1@2 + 16@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-08:58:03.785333 139957508908608 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 17344 Base level 2, inputs: [253278(27MB)], [253239(35MB) 253240(35MB) 253241(35MB) 253242(34MB) 253243(33MB) 253244(34MB) 253245(36MB) 253246(35MB) 253247(35MB) 253248(35MB) 253249(35MB) 253250(35MB) 253251(34MB) 253252(19MB) 253205(34MB) 253209(33MB)] 2024/11/23-08:58:03.791801 139957508908608 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-08:58:03.791831 139957508908608 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-08:58:03.791850 139957508908608 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-08:58:03.791856 139957508908608 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-08:58:03.791865 139957508908608 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-08:58:04.466954 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253282.sst) /dev/shm/ToplingTemp/Topling-6707-DA0oRj 2024/11/23-08:58:05.378898 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253282.sst): first pass time = 0.91's, 147.188'MB/sec 2024/11/23-08:58:06.923400 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253282.sst): old prealloc_size = 147639500, real_size = 32941408 2024/11/23-08:58:06.923919 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253282.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929727 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.36 avg-zval = 33.28 2024/11/23-08:58:06.924801 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253282.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:07.781630 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253282.sst): second pass time = 1.54's, 70.142'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.304'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3542.375'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.12's, 306.323'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36056.257'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.142'MB/sec zip pipeline throughput = 70.142'MB/sec entries = 929727 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.36 avg-zval = 33.28 usrkeys = 929727 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.36 avg-zval = 33.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929727 bytes = 3370272 } seq expand size = 6508089 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5127 value = 3.4966 dict = 1.00 all = 3.6115 } Zip/UnZip{ index = 0.0951 value = 0.2860 dict = 1.00 all = 0.2769 } ---------------------------- total value len = 0.114694 GB avg = 0.123 KB (by entry num) total key len = 0.026032 GB avg = 0.028 KB total ukey len = 0.018595 GB avg = 0.020 KB total ukey num = 0.000929727 Billion total entry num = 0.000929727 Billion write speed all = 0.497820453 MB/sec (with seq num) write speed all = 0.471509043 MB/sec (without seq num) 2024/11/23-08:58:07.833294 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253282: 929727 keys, 37164380 bytes, temperature: kWarm 2024/11/23-08:58:07.833387 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323487833332, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253282, "file_size": 37164380, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3895588561, "table_properties": {"data_size": 30940048, "index_size": 1768768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26032356, "raw_average_key_size": 28, "raw_value_size": 108185462, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929727, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323484, "oldest_key_time": 0, "file_creation_time": 1732323484, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253282, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:07.865171 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253283.sst) /dev/shm/ToplingTemp/Topling-6707-n5Sm0m 2024/11/23-08:58:09.004170 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253283.sst): first pass time = 1.14's, 117.843'MB/sec 2024/11/23-08:58:10.744119 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253283.sst): old prealloc_size = 147639500, real_size = 32719168 2024/11/23-08:58:10.744631 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253283.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929491 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.40 avg-zval = 33.05 2024/11/23-08:58:10.745489 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253283.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:11.793421 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253283.sst): second pass time = 1.74's, 62.209'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 35.206'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3544.671'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.276'MB/sec write SST data time = 0.21's, 177.795'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33563.600'MB/sec zip my value time = 1.74's, unzip length = 0.108'GB zip my value throughput = 62.209'MB/sec zip pipeline throughput = 137.895'MB/sec entries = 929491 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.40 avg-zval = 33.05 usrkeys = 929491 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.40 avg-zval = 33.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929491 bytes = 3369424 } seq expand size = 6506437 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5138 value = 3.5220 dict = 1.00 all = 3.6333 } Zip/UnZip{ index = 0.0951 value = 0.2839 dict = 1.00 all = 0.2752 } ---------------------------- total value len = 0.229392 GB avg = 0.123 KB (by entry num) total key len = 0.052058 GB avg = 0.028 KB total ukey len = 0.037184 GB avg = 0.020 KB total ukey num = 0.001859218 Billion total entry num = 0.001859218 Billion write speed all = 0.981703080 MB/sec (with seq num) write speed all = 0.929823208 MB/sec (without seq num) 2024/11/23-08:58:11.842213 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253283: 929491 keys, 36941300 bytes, temperature: kWarm 2024/11/23-08:58:11.842268 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323491842238, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253283, "file_size": 36941300, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3895589917, "table_properties": {"data_size": 30718512, "index_size": 1768128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26025748, "raw_average_key_size": 28, "raw_value_size": 108192110, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929491, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323487, "oldest_key_time": 0, "file_creation_time": 1732323487, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253283, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:11.874607 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253284.sst) /dev/shm/ToplingTemp/Topling-6707-iHXU0m 2024/11/23-08:58:12.946402 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253284.sst): first pass time = 1.07's, 125.231'MB/sec 2024/11/23-08:58:14.491680 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253284.sst): old prealloc_size = 147639500, real_size = 32739392 2024/11/23-08:58:14.492199 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253284.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929659 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.37 avg-zval = 33.06 2024/11/23-08:58:14.493015 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253284.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:15.514590 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253284.sst): second pass time = 1.54's, 70.046'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 36.135'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3610.362'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.261'MB/sec write SST data time = 0.13's, 280.131'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34557.157'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.046'MB/sec zip pipeline throughput = 70.046'MB/sec entries = 929659 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.37 avg-zval = 33.06 usrkeys = 929659 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.37 avg-zval = 33.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929659 bytes = 3370032 } seq expand size = 6507613 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5116 value = 3.5197 dict = 1.00 all = 3.6312 } Zip/UnZip{ index = 0.0951 value = 0.2841 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 0.344087 GB avg = 0.123 KB (by entry num) total key len = 0.078089 GB avg = 0.028 KB total ukey len = 0.055778 GB avg = 0.020 KB total ukey num = 0.002788877 Billion total entry num = 0.002788877 Billion write speed all = 1.453687561 MB/sec (with seq num) write speed all = 1.376863495 MB/sec (without seq num) 2024/11/23-08:58:15.562073 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253284: 929659 keys, 36962052 bytes, temperature: kWarm 2024/11/23-08:58:15.562139 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323495562105, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253284, "file_size": 36962052, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 3895590090, "table_properties": {"data_size": 30737968, "index_size": 1768832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26030452, "raw_average_key_size": 28, "raw_value_size": 108187334, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929659, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323491, "oldest_key_time": 0, "file_creation_time": 1732323491, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253284, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:15.599828 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253285.sst) /dev/shm/ToplingTemp/Topling-6707-fW3tjk 2024/11/23-08:58:16.740500 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253285.sst): first pass time = 1.14's, 117.670'MB/sec 2024/11/23-08:58:18.143562 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253285.sst): old prealloc_size = 147639500, real_size = 32289360 2024/11/23-08:58:18.144104 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253285.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931252 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 116.13 avg-zval = 32.52 2024/11/23-08:58:18.144898 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253285.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:19.228954 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253285.sst): second pass time = 1.40's, 77.113'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.09's, 33.537'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3361.798'MB/sec (index lex order gen) rebuild zvType time = 0.95's, 0.246'MB/sec write SST data time = 0.14's, 261.747'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31763.146'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.113'MB/sec zip pipeline throughput = 77.113'MB/sec entries = 931252 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 32.52 usrkeys = 931252 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 32.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931252 bytes = 3259392 } seq expand size = 6518764 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5144 value = 3.5708 dict = 1.00 all = 3.6873 } Zip/UnZip{ index = 0.0951 value = 0.2800 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 0.458749 GB avg = 0.123 KB (by entry num) total key len = 0.104164 GB avg = 0.028 KB total ukey len = 0.074403 GB avg = 0.020 KB total ukey num = 0.003720129 Billion total entry num = 0.003720129 Billion write speed all = 1.913811894 MB/sec (with seq num) write speed all = 1.812629112 MB/sec (without seq num) 2024/11/23-08:58:19.273578 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253285: 931252 keys, 36400008 bytes, temperature: kWarm 2024/11/23-08:58:19.273656 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323499273610, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253285, "file_size": 36400008, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 3895589131, "table_properties": {"data_size": 30284992, "index_size": 1771392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26075056, "raw_average_key_size": 28, "raw_value_size": 108142724, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931252, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323495, "oldest_key_time": 0, "file_creation_time": 1732323495, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253285, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:19.303477 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253286.sst) /dev/shm/ToplingTemp/Topling-6707-kSy5pn 2024/11/23-08:58:20.351086 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253286.sst): first pass time = 1.05's, 128.123'MB/sec 2024/11/23-08:58:21.672923 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253286.sst): old prealloc_size = 147639500, real_size = 31614512 2024/11/23-08:58:21.673439 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253286.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932031 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 116.01 avg-zval = 31.76 2024/11/23-08:58:21.674324 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253286.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:22.559966 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253286.sst): second pass time = 1.32's, 81.834'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.146'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3585.994'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.308'MB/sec write SST data time = 0.13's, 275.484'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 40444.753'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.834'MB/sec zip pipeline throughput = 82.326'MB/sec entries = 932031 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.01 avg-zval = 31.76 usrkeys = 932031 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.01 avg-zval = 31.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932031 bytes = 3145616 } seq expand size = 6524217 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4462 value = 3.6531 dict = 1.00 all = 3.7690 } Zip/UnZip{ index = 0.0957 value = 0.2737 dict = 1.00 all = 0.2653 } ---------------------------- total value len = 0.573394 GB avg = 0.123 KB (by entry num) total key len = 0.130260 GB avg = 0.028 KB total ukey len = 0.093043 GB avg = 0.020 KB total ukey num = 0.004652160 Billion total entry num = 0.004652160 Billion write speed all = 2.365523024 MB/sec (with seq num) write speed all = 2.240407108 MB/sec (without seq num) 2024/11/23-08:58:22.603695 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253286: 932031 keys, 35610618 bytes, temperature: kWarm 2024/11/23-08:58:22.603761 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323502603727, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253286, "file_size": 35610618, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 3895590472, "table_properties": {"data_size": 29596896, "index_size": 1784448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26096868, "raw_average_key_size": 28, "raw_value_size": 108120914, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 932031, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323499, "oldest_key_time": 0, "file_creation_time": 1732323499, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253286, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:22.637267 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253287.sst) /dev/shm/ToplingTemp/Topling-6707-1oUmtl 2024/11/23-08:58:23.862580 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253287.sst): first pass time = 1.23's, 109.542'MB/sec 2024/11/23-08:58:25.700218 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253287.sst): old prealloc_size = 147639500, real_size = 35148112 2024/11/23-08:58:25.700845 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253287.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1082674 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1039 GB zip-val = 0.0328 GB avg-val = 95.97 avg-zval = 30.29 2024/11/23-08:58:25.701042 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253287.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:26.446226 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253287.sst): second pass time = 1.84's, 56.565'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.75's, 48.617'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3436.339'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.493'MB/sec write SST data time = 0.20's, 184.293'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.032'MB, throughput = 34154.253'MB/sec zip my value time = 1.84's, unzip length = 0.104'GB zip my value throughput = 56.565'MB/sec zip pipeline throughput = 109.582'MB/sec entries = 1082674 avg-key = 28.00 avg-zkey = 1.93 avg-val = 95.97 avg-zval = 30.29 usrkeys = 1082674 avg-key = 20.00 avg-zkey = 1.93 avg-val = 96.35 avg-zval = 30.29 TagRS{ kinds = RS_Key0_TagN bytes = 169280 } TagArray{ size = 59007 bytes = 177040 } seq expand size = 413049 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0217 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0328 GB dict = 1.03 MB all = 0.0363 GB } UnZip/Zip{ index = 10.3803 value = 3.1686 dict = 1.00 all = 3.7008 } Zip/UnZip{ index = 0.0963 value = 0.3156 dict = 1.00 all = 0.2702 } ---------------------------- total value len = 0.677710 GB avg = 0.118 KB (by entry num) total key len = 0.160575 GB avg = 0.028 KB total ukey len = 0.114697 GB avg = 0.020 KB total ukey num = 0.005734834 Billion total entry num = 0.005734834 Billion write speed all = 2.781777818 MB/sec (with seq num) write speed all = 2.629533323 MB/sec (without seq num) 2024/11/23-08:58:26.496355 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253287: 1082674 keys, 36267011 bytes, temperature: kWarm 2024/11/23-08:58:26.496431 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323506496385, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253287, "file_size": 36267011, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3895588803, "table_properties": {"data_size": 32791232, "index_size": 2086016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30314872, "raw_average_key_size": 28, "raw_value_size": 103902909, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1082674, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323502, "oldest_key_time": 0, "file_creation_time": 1732323502, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253287, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:26.557847 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253288.sst) /dev/shm/ToplingTemp/Topling-6707-MCN8bn 2024/11/23-08:58:27.488263 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253288.sst): first pass time = 0.93's, 144.264'MB/sec 2024/11/23-08:58:28.820147 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253288.sst): old prealloc_size = 147639500, real_size = 35025680 2024/11/23-08:58:28.820652 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253288.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922364 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0330 GB avg-val = 117.52 avg-zval = 35.81 2024/11/23-08:58:28.821230 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253288.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:30.296242 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253288.sst): second pass time = 1.33's, 81.424'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.48's, 25.828'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3621.814'MB/sec (index lex order gen) rebuild zvType time = 1.34's, 0.173'MB/sec write SST data time = 0.14's, 272.525'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 26953.322'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.424'MB/sec zip pipeline throughput = 81.424'MB/sec entries = 922364 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.52 avg-zval = 35.81 usrkeys = 922364 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.75 avg-zval = 35.81 TagRS{ kinds = RS_Key0_TagN bytes = 144200 } TagArray{ size = 557628 bytes = 2091120 } seq expand size = 3903396 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4442 value = 3.2818 dict = 1.00 all = 3.5206 } Zip/UnZip{ index = 0.0957 value = 0.3047 dict = 1.00 all = 0.2840 } ---------------------------- total value len = 0.790005 GB avg = 0.119 KB (by entry num) total key len = 0.186402 GB avg = 0.028 KB total ukey len = 0.133144 GB avg = 0.020 KB total ukey num = 0.006657198 Billion total entry num = 0.006657198 Billion write speed all = 3.199247979 MB/sec (with seq num) write speed all = 3.024746616 MB/sec (without seq num) 2024/11/23-08:58:30.344971 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253288: 922364 keys, 38123142 bytes, temperature: kWarm 2024/11/23-08:58:30.345090 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323510345029, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253288, "file_size": 38123142, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3843797943, "table_properties": {"data_size": 33028608, "index_size": 1766272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25826192, "raw_average_key_size": 28, "raw_value_size": 108391664, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922364, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323506, "oldest_key_time": 0, "file_creation_time": 1732323506, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253288, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:30.393895 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253289.sst) /dev/shm/ToplingTemp/Topling-6707-XbTceo 2024/11/23-08:58:31.673115 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253289.sst): first pass time = 1.28's, 104.927'MB/sec 2024/11/23-08:58:34.446699 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253289.sst): old prealloc_size = 147639500, real_size = 32935184 2024/11/23-08:58:34.447327 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253289.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929360 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.42 avg-zval = 33.29 2024/11/23-08:58:34.449314 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253289.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:35.652485 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253289.sst): second pass time = 2.77's, 39.021'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.21's, 30.817'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2908.254'MB/sec (index lex order gen) rebuild zvType time = 1.07's, 0.217'MB/sec write SST data time = 0.14's, 273.801'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30333.493'MB/sec zip my value time = 2.77's, unzip length = 0.108'GB zip my value throughput = 39.021'MB/sec zip pipeline throughput = 135.215'MB/sec entries = 929360 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.42 avg-zval = 33.29 usrkeys = 929360 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.42 avg-zval = 33.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929360 bytes = 3368944 } seq expand size = 6505520 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5127 value = 3.4976 dict = 1.00 all = 3.6122 } Zip/UnZip{ index = 0.0951 value = 0.2859 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 0.904706 GB avg = 0.119 KB (by entry num) total key len = 0.212424 GB avg = 0.028 KB total ukey len = 0.151731 GB avg = 0.020 KB total ukey num = 0.007586558 Billion total entry num = 0.007586558 Billion write speed all = 3.597204971 MB/sec (with seq num) write speed all = 3.401772647 MB/sec (without seq num) 2024/11/23-08:58:35.702027 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253289: 929360 keys, 37156880 bytes, temperature: kWarm 2024/11/23-08:58:35.702090 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323515702057, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253289, "file_size": 37156880, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3895589931, "table_properties": {"data_size": 30934592, "index_size": 1768064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26022080, "raw_average_key_size": 28, "raw_value_size": 108195772, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929360, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323510, "oldest_key_time": 0, "file_creation_time": 1732323510, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253289, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:35.737777 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253290.sst) /dev/shm/ToplingTemp/Topling-6707-wJ60Fj 2024/11/23-08:58:36.746296 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253290.sst): first pass time = 1.01's, 133.092'MB/sec 2024/11/23-08:58:39.726593 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253290.sst): old prealloc_size = 147639500, real_size = 32985264 2024/11/23-08:58:39.727110 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253290.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929311 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.43 avg-zval = 33.34 2024/11/23-08:58:39.728800 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253290.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:40.629205 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253290.sst): second pass time = 2.98's, 36.312'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.965'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3527.594'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.12's, 305.073'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34267.556'MB/sec zip my value time = 2.98's, unzip length = 0.108'GB zip my value throughput = 36.312'MB/sec zip pipeline throughput = 122.412'MB/sec entries = 929311 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.43 avg-zval = 33.34 usrkeys = 929311 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.43 avg-zval = 33.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929311 bytes = 3136432 } seq expand size = 6505177 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5110 value = 3.4920 dict = 1.00 all = 3.6300 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 1.019408 GB avg = 0.120 KB (by entry num) total key len = 0.238444 GB avg = 0.028 KB total ukey len = 0.170317 GB avg = 0.020 KB total ukey num = 0.008515869 Billion total entry num = 0.008515869 Billion write speed all = 3.986454784 MB/sec (with seq num) write speed all = 3.770543146 MB/sec (without seq num) 2024/11/23-08:58:40.674287 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253290: 929311 keys, 36974612 bytes, temperature: kWarm 2024/11/23-08:58:40.674353 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323520674318, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253290, "file_size": 36974612, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 3895589946, "table_properties": {"data_size": 30984480, "index_size": 1768256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26020708, "raw_average_key_size": 28, "raw_value_size": 108197114, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929311, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323515, "oldest_key_time": 0, "file_creation_time": 1732323515, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253290, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:40.707945 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253291.sst) /dev/shm/ToplingTemp/Topling-6707-X03iLk 2024/11/23-08:58:41.659317 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253291.sst): first pass time = 0.95's, 141.088'MB/sec 2024/11/23-08:58:42.923685 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253291.sst): old prealloc_size = 147639500, real_size = 33331456 2024/11/23-08:58:42.924202 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253291.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929406 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.41 avg-zval = 33.71 2024/11/23-08:58:42.925025 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253291.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:43.773211 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253291.sst): second pass time = 1.26's, 85.623'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.344'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3583.157'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 309.931'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 21970.443'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.623'MB/sec zip pipeline throughput = 85.623'MB/sec entries = 929406 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.41 avg-zval = 33.71 usrkeys = 929406 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.41 avg-zval = 33.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929406 bytes = 3485280 } seq expand size = 6505842 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5117 value = 3.4533 dict = 1.00 all = 3.5630 } Zip/UnZip{ index = 0.0951 value = 0.2896 dict = 1.00 all = 0.2807 } ---------------------------- total value len = 1.134109 GB avg = 0.120 KB (by entry num) total key len = 0.264468 GB avg = 0.028 KB total ukey len = 0.188906 GB avg = 0.020 KB total ukey num = 0.009445275 Billion total entry num = 0.009445275 Billion write speed all = 4.388714421 MB/sec (with seq num) write speed all = 4.151601204 MB/sec (without seq num) 2024/11/23-08:58:43.819381 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253291: 929406 keys, 37669544 bytes, temperature: kWarm 2024/11/23-08:58:43.819446 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323523819411, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253291, "file_size": 37669544, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3895589963, "table_properties": {"data_size": 31330608, "index_size": 1768320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26023368, "raw_average_key_size": 28, "raw_value_size": 108194452, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929406, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323520, "oldest_key_time": 0, "file_creation_time": 1732323520, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253291, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:43.853802 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253292.sst) /dev/shm/ToplingTemp/Topling-6707-1RhHLj 2024/11/23-08:58:44.695172 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253292.sst): first pass time = 0.84's, 159.530'MB/sec 2024/11/23-08:58:45.972295 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253292.sst): old prealloc_size = 147639500, real_size = 33036256 2024/11/23-08:58:45.972807 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253292.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929656 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.37 avg-zval = 33.38 2024/11/23-08:58:45.973577 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253292.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:46.829240 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253292.sst): second pass time = 1.28's, 84.765'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.345'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3595.595'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.12's, 307.567'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35841.158'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.765'MB/sec zip pipeline throughput = 84.765'MB/sec entries = 929656 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.37 avg-zval = 33.38 usrkeys = 929656 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.37 avg-zval = 33.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929656 bytes = 3253808 } seq expand size = 6507592 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5130 value = 3.4860 dict = 1.00 all = 3.6136 } Zip/UnZip{ index = 0.0951 value = 0.2869 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 1.248804 GB avg = 0.120 KB (by entry num) total key len = 0.290498 GB avg = 0.028 KB total ukey len = 0.207499 GB avg = 0.020 KB total ukey num = 0.010374931 Billion total entry num = 0.010374931 Billion write speed all = 4.784427079 MB/sec (with seq num) write speed all = 4.526449839 MB/sec (without seq num) 2024/11/23-08:58:46.874156 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253292: 929656 keys, 37142904 bytes, temperature: kWarm 2024/11/23-08:58:46.874226 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323526874190, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253292, "file_size": 37142904, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3895589982, "table_properties": {"data_size": 31035088, "index_size": 1768576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26030368, "raw_average_key_size": 28, "raw_value_size": 108187496, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929656, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323523, "oldest_key_time": 0, "file_creation_time": 1732323523, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253292, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:46.908257 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253293.sst) /dev/shm/ToplingTemp/Topling-6707-pgyecm 2024/11/23-08:58:47.734441 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253293.sst): first pass time = 0.83's, 162.462'MB/sec 2024/11/23-08:58:49.013850 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253293.sst): old prealloc_size = 147639500, real_size = 33174720 2024/11/23-08:58:49.014382 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253293.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929388 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.42 avg-zval = 33.54 2024/11/23-08:58:49.015175 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253293.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:49.864787 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253293.sst): second pass time = 1.28's, 84.616'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.949'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3468.334'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 308.734'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33082.973'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.616'MB/sec zip pipeline throughput = 84.616'MB/sec entries = 929388 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.42 avg-zval = 33.54 usrkeys = 929388 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.42 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929388 bytes = 3369040 } seq expand size = 6505716 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5108 value = 3.4707 dict = 1.00 all = 3.5890 } Zip/UnZip{ index = 0.0951 value = 0.2881 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 1.363504 GB avg = 0.121 KB (by entry num) total key len = 0.316521 GB avg = 0.028 KB total ukey len = 0.226086 GB avg = 0.020 KB total ukey num = 0.011304319 Billion total entry num = 0.011304319 Billion write speed all = 5.173013834 MB/sec (with seq num) write speed all = 4.894554224 MB/sec (without seq num) 2024/11/23-08:58:49.909136 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253293: 929388 keys, 37396596 bytes, temperature: kWarm 2024/11/23-08:58:49.909200 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323529909165, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253293, "file_size": 37396596, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3895589997, "table_properties": {"data_size": 31173744, "index_size": 1768448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26022864, "raw_average_key_size": 28, "raw_value_size": 108194996, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929388, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323526, "oldest_key_time": 0, "file_creation_time": 1732323526, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253293, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:49.942552 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253294.sst) /dev/shm/ToplingTemp/Topling-6707-MWeuIn 2024/11/23-08:58:50.847757 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253294.sst): first pass time = 0.91's, 148.280'MB/sec 2024/11/23-08:58:52.103147 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253294.sst): old prealloc_size = 147639500, real_size = 31339424 2024/11/23-08:58:52.103664 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253294.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930341 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0293 GB avg-val = 116.27 avg-zval = 31.53 2024/11/23-08:58:52.104451 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253294.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:52.923336 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253294.sst): second pass time = 1.25's, 86.211'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.362'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3571.116'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.12's, 293.219'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33598.692'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.211'MB/sec zip pipeline throughput = 86.211'MB/sec entries = 930341 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.27 avg-zval = 31.53 usrkeys = 930341 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.27 avg-zval = 31.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930341 bytes = 3372496 } seq expand size = 6512387 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5128 value = 3.6871 dict = 1.00 all = 3.7740 } Zip/UnZip{ index = 0.0951 value = 0.2712 dict = 1.00 all = 0.2650 } ---------------------------- total value len = 1.478185 GB avg = 0.121 KB (by entry num) total key len = 0.342570 GB avg = 0.028 KB total ukey len = 0.244693 GB avg = 0.020 KB total ukey num = 0.012234660 Billion total entry num = 0.012234660 Billion write speed all = 5.554034120 MB/sec (with seq num) write speed all = 5.255469181 MB/sec (without seq num) 2024/11/23-08:58:52.966099 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253294: 930341 keys, 35563990 bytes, temperature: kWarm 2024/11/23-08:58:52.966165 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323532966131, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253294, "file_size": 35563990, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3895590023, "table_properties": {"data_size": 29336720, "index_size": 1769920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26049548, "raw_average_key_size": 28, "raw_value_size": 108168266, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930341, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323529, "oldest_key_time": 0, "file_creation_time": 1732323529, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253294, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:53.003816 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253295.sst) /dev/shm/ToplingTemp/Topling-6707-uryQHl 2024/11/23-08:58:54.187684 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253295.sst): first pass time = 1.18's, 113.375'MB/sec 2024/11/23-08:58:55.463495 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253295.sst): old prealloc_size = 147639500, real_size = 33280720 2024/11/23-08:58:55.464049 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253295.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934204 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0313 GB avg-val = 115.67 avg-zval = 33.47 2024/11/23-08:58:55.464875 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253295.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:56.305908 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253295.sst): second pass time = 1.28's, 84.749'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.537'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3327.034'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.12's, 306.735'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34695.202'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.749'MB/sec zip pipeline throughput = 84.749'MB/sec entries = 934204 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.67 avg-zval = 33.47 usrkeys = 934204 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.67 avg-zval = 33.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934204 bytes = 3386512 } seq expand size = 6539428 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5135 value = 3.4557 dict = 1.00 all = 3.5775 } Zip/UnZip{ index = 0.0951 value = 0.2894 dict = 1.00 all = 0.2795 } ---------------------------- total value len = 1.592785 GB avg = 0.121 KB (by entry num) total key len = 0.368728 GB avg = 0.028 KB total ukey len = 0.263377 GB avg = 0.020 KB total ukey num = 0.013168864 Billion total entry num = 0.013168864 Billion write speed all = 5.922292840 MB/sec (with seq num) write speed all = 5.604212344 MB/sec (without seq num) 2024/11/23-08:58:56.352152 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253295: 934204 keys, 37517652 bytes, temperature: kWarm 2024/11/23-08:58:56.352219 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323536352184, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253295, "file_size": 37517652, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3895590147, "table_properties": {"data_size": 31269824, "index_size": 1777152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26157712, "raw_average_key_size": 28, "raw_value_size": 108060132, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934204, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323532, "oldest_key_time": 0, "file_creation_time": 1732323532, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253295, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:56.385468 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253296.sst) /dev/shm/ToplingTemp/Topling-6707-GmEvqm 2024/11/23-08:58:57.311046 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253296.sst): first pass time = 0.93's, 145.014'MB/sec 2024/11/23-08:58:58.579426 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253296.sst): old prealloc_size = 147639500, real_size = 32958656 2024/11/23-08:58:58.579944 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253296.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929041 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.47 avg-zval = 33.32 2024/11/23-08:58:58.580739 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253296.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:58:59.426955 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253296.sst): second pass time = 1.27's, 85.364'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.869'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3586.249'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.12's, 305.487'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33659.308'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.364'MB/sec zip pipeline throughput = 85.364'MB/sec entries = 929041 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.47 avg-zval = 33.32 usrkeys = 929041 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.47 avg-zval = 33.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929041 bytes = 3367792 } seq expand size = 6503287 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5114 value = 3.4952 dict = 1.00 all = 3.6100 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 1.707493 GB avg = 0.121 KB (by entry num) total key len = 0.394741 GB avg = 0.028 KB total ukey len = 0.281958 GB avg = 0.020 KB total ukey num = 0.014097905 Billion total entry num = 0.014097905 Billion write speed all = 6.287912496 MB/sec (with seq num) write speed all = 5.950570805 MB/sec (without seq num) 2024/11/23-08:58:59.471132 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253296: 929041 keys, 37179438 bytes, temperature: kWarm 2024/11/23-08:58:59.471200 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323539471165, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253296, "file_size": 37179438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3895590049, "table_properties": {"data_size": 30958512, "index_size": 1767680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26013148, "raw_average_key_size": 28, "raw_value_size": 108204718, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929041, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323536, "oldest_key_time": 0, "file_creation_time": 1732323536, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253296, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:58:59.504592 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253297.sst) /dev/shm/ToplingTemp/Topling-6707-ybtVhk 2024/11/23-08:58:59.870690 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253297.sst): first pass time = 0.37's, 249.571'MB/sec 2024/11/23-08:59:00.777348 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253297.sst): old prealloc_size = 147639500, real_size = 22288576 2024/11/23-08:59:00.777751 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253297.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 630915 prefix = 4 raw-key = 0.0101 GB zip-key = 0.0012 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0737 GB zip-val = 0.0209 GB avg-val = 116.80 avg-zval = 33.16 2024/11/23-08:59:00.778293 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253297.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0016 GB, waited 0.000 sec, Key+Value = 0.091 GB 2024/11/23-08:59:01.365090 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00/253297.sst): second pass time = 0.91's, 81.342'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.59's, 42.677'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3176.063'MB/sec (index lex order gen) rebuild zvType time = 0.50's, 0.313'MB/sec write SST data time = 0.08's, 301.158'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.740'MB, throughput = 22057.662'MB/sec zip my value time = 0.91's, unzip length = 0.074'GB zip my value throughput = 81.342'MB/sec zip pipeline throughput = 81.342'MB/sec entries = 630915 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.80 avg-zval = 33.16 usrkeys = 630915 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.80 avg-zval = 33.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 630915 bytes = 2208224 } seq expand size = 4416405 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0126 GB value = 0.0737 GB dict = 0.74 MB all = 0.0914 GB } __ZipSize{ index = 0.0012 GB value = 0.0209 GB dict = 0.74 MB all = 0.0251 GB } UnZip/Zip{ index = 10.4423 value = 3.5221 dict = 1.00 all = 3.6423 } Zip/UnZip{ index = 0.0958 value = 0.2839 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 1.785600 GB avg = 0.121 KB (by entry num) total key len = 0.412407 GB avg = 0.028 KB total ukey len = 0.294576 GB avg = 0.020 KB total ukey num = 0.014728820 Billion total entry num = 0.014728820 Billion write speed all = 6.536483017 MB/sec (with seq num) write speed all = 6.186075822 MB/sec (without seq num) 2024/11/23-08:59:01.400165 139957508908608 [compaction_job.cc:1948] [default] [JOB 23096] Generated table #253297: 630915 keys, 25081974 bytes, temperature: kWarm 2024/11/23-08:59:01.400231 139957508908608 EVENT_LOG_v1 {"time_micros": 1732323541400197, "cf_name": "default", "job": 23096, "event": "table_file_creation", "file_number": 253297, "file_size": 25081974, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3748810390, "largest_seqno": 3895307533, "table_properties": {"data_size": 20922288, "index_size": 1208384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 17665620, "raw_average_key_size": 28, "raw_value_size": 73691018, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 630915, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323539, "oldest_key_time": 0, "file_creation_time": 1732323539, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253297, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:59:01.419566 139957508908608 [compaction_job.cc:703] job-23096: subcompact[0], size: 0.579152 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23096/att-00 [253282,253283,253284,253285,253286,253287,253288,253289,253290,253291,253292,253293,253294,253295,253296,253297] 2024/11/23-08:59:01.531746 139957508908608 (Original Log Time 2024/11/23-08:58:03.785406) EVENT_LOG_v1 {"time_micros": 1732323483785352, "job": 23096, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [253278], "files_L3": [253239, 253240, 253241, 253242, 253243, 253244, 253245, 253246, 253247, 253248, 253249, 253250, 253251, 253252, 253205, 253209], "score": 0.991332, "input_data_size": 599285727, "oldest_snapshot_seqno": 3895605398} 2024/11/23-08:59:01.531750 139957508908608 (Original Log Time 2024/11/23-08:59:01.530491) [compaction_job.cc:2022] [default] [JOB 23096] Compacted 1@2 + 16@3 files to L3 => 579152101 bytes 2024/11/23-08:59:01.531751 139957508908608 (Original Log Time 2024/11/23-08:59:01.531681) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.4 rd, 10.0 wr, level 3, files in(1, 16) out(16 +0 blob) MB in(27.1, 544.4 +0.0 blob) out(552.3 +0.0 blob), read-write-amplify(41.5) write-amplify(20.4) OK, records in: 14811145, records dropped: 82325 output_compression: Snappy 2024/11/23-08:59:01.531757 139957508908608 (Original Log Time 2024/11/23-08:59:01.531726) EVENT_LOG_v1 {"time_micros": 1732323541531705, "cf": "default", "job": 23096, "event": "compaction_finished", "compaction_time_micros": 57634169, "compaction_time_cpu_micros": 24327916, "output_level": 3, "num_output_files": 16, "total_output_size": 579152101, "num_input_records": 14811145, "num_output_records": 14728820, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 451917260, "file_range_sync_nanos": 0, "file_fsync_nanos": 10490, "file_prepare_write_nanos": 42585, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/23-08:59:01.671289 139957508908608 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732323483,"executesMs":57884,"compactionJobId":23096,"attempt":0,"compactionInputRawBytes":2116314191,"compactionInputZipBytes":599285727,"compactionOutputRawBytes":0,"compactionOutputZipBytes":579152101,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-f6b722fd99b2c43ab78a9c4177323841-2d26d95c8fddc01f-00"}}, response = 2024/11/23-08:59:01.671377 139957508908608 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-23096/att-00: olev 3, work 57.884 s, result 7.682 ms, install 1.283 ms, input{raw 1.971 GiB zip 571.523 MiB}