2024/11/23-18:04:28.583071 139957374625344 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00: fork to child time = 0.011060 sec 2024/11/23-18:04:28.583863 139957374625344 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00 2024/11/23-18:04:28.592798 139957374625344 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00 2024/11/23-18:04:28.593749 139957374625344 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-18:04:28.594045 139957374625344 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-18:04:28.646873 139957374625344 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32639/att-00/MANIFEST-206347 2024/11/23-18:04:28.695639 139957374625344 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32639/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 273843, last_sequence is 4169464266, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 273709 2024/11/23-18:04:28.695650 139957374625344 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-18:04:28.695822 139957374625344 [version_set.cc:5721] Creating manifest 273843 2024/11/23-18:04:28.732647 139957374625344 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-18:04:28.756510 139957374625344 [compaction_job.cc:2437] [default] [JOB 32639] Compacting 1@2 + 18@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-18:04:28.756538 139957374625344 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24436 Base level 2, inputs: [273837(25MB)], [273752(37MB) 273753(37MB) 273792(37MB) 273793(38MB) 273794(35MB) 273795(35MB) 273796(37MB) 273797(35MB) 273798(37MB) 273799(34MB) 273800(35MB) 273801(37MB) 273802(35MB) 273803(37MB) 273804(35MB) 273805(35MB) 273806(36MB) 273807(37MB)] 2024/11/23-18:04:28.766766 139957374625344 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:04:28.766810 139957374625344 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:04:28.766838 139957374625344 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:04:28.766847 139957374625344 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:04:28.766866 139957374625344 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:04:29.453203 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273844.sst) /dev/shm/ToplingTemp/Topling-143155-bjnDBU 2024/11/23-18:04:30.831205 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273844.sst): first pass time = 1.38's, 97.404'MB/sec 2024/11/23-18:04:32.852498 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273844.sst): old prealloc_size = 147639500, real_size = 35124128 2024/11/23-18:04:32.853092 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273844.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927661 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.68 avg-zval = 35.71 2024/11/23-18:04:32.853875 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273844.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:04:33.936244 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273844.sst): second pass time = 2.02's, 53.580'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 36.087'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3078.934'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.17's, 226.754'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 40362.244'MB/sec zip my value time = 2.02's, unzip length = 0.108'GB zip my value throughput = 53.580'MB/sec zip pipeline throughput = 53.580'MB/sec entries = 927661 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 35.71 usrkeys = 927661 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 35.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927661 bytes = 3130864 } seq expand size = 6493627 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5141 value = 3.2675 dict = 1.00 all = 3.4319 } Zip/UnZip{ index = 0.0951 value = 0.3060 dict = 1.00 all = 0.2914 } ---------------------------- total value len = 8.963357 GB avg = 0.168 KB (by entry num) total key len = 1.466796 GB avg = 0.028 KB total ukey len = 1.016431 GB avg = 0.015 KB total ukey num = 0.066478096 Billion total entry num = 0.053285825 Billion write speed all = 0.316432270 MB/sec (with seq num) write speed all = 0.303499492 MB/sec (without seq num) 2024/11/23-18:04:33.988199 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273844: 927661 keys, 39109014 bytes, temperature: kWarm 2024/11/23-18:04:33.988291 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356273988231, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273844, "file_size": 39109014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4094024708, "largest_seqno": 4169435967, "table_properties": {"data_size": 33127440, "index_size": 1764608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25974508, "raw_average_key_size": 28, "raw_value_size": 108243362, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927661, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356269, "oldest_key_time": 0, "file_creation_time": 1732356269, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273844, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:04:34.027768 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273845.sst) /dev/shm/ToplingTemp/Topling-143155-ztJMmW 2024/11/23-18:04:35.004351 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273845.sst): first pass time = 0.98's, 137.441'MB/sec 2024/11/23-18:04:37.942945 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273845.sst): old prealloc_size = 147639500, real_size = 35260448 2024/11/23-18:04:37.943555 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273845.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926563 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 116.86 avg-zval = 35.89 2024/11/23-18:04:37.944406 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273845.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:04:39.092844 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273845.sst): second pass time = 2.94's, 36.854'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.15's, 34.128'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3121.196'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.32's, 122.267'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28607.001'MB/sec zip my value time = 2.94's, unzip length = 0.108'GB zip my value throughput = 36.854'MB/sec zip pipeline throughput = 112.986'MB/sec entries = 926563 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.86 avg-zval = 35.89 usrkeys = 926563 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.86 avg-zval = 35.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926563 bytes = 3127168 } seq expand size = 6485941 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.4444 value = 3.2559 dict = 1.00 all = 3.4202 } Zip/UnZip{ index = 0.0957 value = 0.3071 dict = 1.00 all = 0.2924 } ---------------------------- total value len = 9.078117 GB avg = 0.167 KB (by entry num) total key len = 1.492739 GB avg = 0.028 KB total ukey len = 1.034962 GB avg = 0.015 KB total ukey num = 0.067404659 Billion total entry num = 0.054212388 Billion write speed all = 0.320650807 MB/sec (with seq num) write speed all = 0.307495206 MB/sec (without seq num) 2024/11/23-18:04:39.149430 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273845: 926563 keys, 39242244 bytes, temperature: kWarm 2024/11/23-18:04:39.149517 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356279149470, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273845, "file_size": 39242244, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668904, "largest_seqno": 4169435877, "table_properties": {"data_size": 33254352, "index_size": 1774272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25943764, "raw_average_key_size": 28, "raw_value_size": 108274010, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926563, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356274, "oldest_key_time": 0, "file_creation_time": 1732356274, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273845, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:04:39.211783 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273846.sst) /dev/shm/ToplingTemp/Topling-143155-cUifZS 2024/11/23-18:04:40.477806 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273846.sst): first pass time = 1.27's, 106.020'MB/sec 2024/11/23-18:04:42.812378 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273846.sst): old prealloc_size = 147639500, real_size = 35407600 2024/11/23-18:04:42.812961 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273846.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928210 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0334 GB avg-val = 116.60 avg-zval = 35.98 2024/11/23-18:04:42.813763 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273846.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:04:43.923730 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273846.sst): second pass time = 2.33's, 46.372'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 35.344'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3164.944'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.264'MB/sec write SST data time = 0.23's, 168.340'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29236.992'MB/sec zip my value time = 2.33's, unzip length = 0.108'GB zip my value throughput = 46.372'MB/sec zip pipeline throughput = 113.286'MB/sec entries = 928210 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.60 avg-zval = 35.98 usrkeys = 928210 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.60 avg-zval = 35.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928210 bytes = 3016704 } seq expand size = 6497470 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4468 value = 3.2405 dict = 1.00 all = 3.4171 } Zip/UnZip{ index = 0.0957 value = 0.3086 dict = 1.00 all = 0.2926 } ---------------------------- total value len = 9.192842 GB avg = 0.167 KB (by entry num) total key len = 1.518729 GB avg = 0.028 KB total ukey len = 1.053527 GB avg = 0.015 KB total ukey num = 0.068332869 Billion total entry num = 0.055140598 Billion write speed all = 0.324871583 MB/sec (with seq num) write speed all = 0.311492695 MB/sec (without seq num) 2024/11/23-18:04:43.976230 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273846: 928210 keys, 39278052 bytes, temperature: kWarm 2024/11/23-18:04:43.976296 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356283976263, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273846, "file_size": 39278052, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4131604997, "largest_seqno": 4169436314, "table_properties": {"data_size": 33398368, "index_size": 1777024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989880, "raw_average_key_size": 28, "raw_value_size": 108227868, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928210, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356279, "oldest_key_time": 0, "file_creation_time": 1732356279, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273846, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:04:44.021433 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273847.sst) /dev/shm/ToplingTemp/Topling-143155-XDsn3T 2024/11/23-18:04:45.673933 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273847.sst): first pass time = 1.65's, 81.223'MB/sec 2024/11/23-18:04:47.281656 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273847.sst): old prealloc_size = 147639500, real_size = 36694720 2024/11/23-18:04:47.282194 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273847.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935481 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0347 GB avg-val = 115.47 avg-zval = 37.05 2024/11/23-18:04:47.282987 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273847.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:04:48.271885 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273847.sst): second pass time = 1.61's, 67.230'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 41.217'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3461.522'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.272'MB/sec write SST data time = 0.13's, 311.824'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34613.936'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.230'MB/sec zip pipeline throughput = 67.230'MB/sec entries = 935481 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.47 avg-zval = 37.05 usrkeys = 935481 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.70 avg-zval = 37.05 TagRS{ kinds = RS_Key0_TagN bytes = 146280 } TagArray{ size = 831754 bytes = 3119088 } seq expand size = 5822278 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0408 GB } UnZip/Zip{ index = 10.3773 value = 3.1169 dict = 1.00 all = 3.2886 } Zip/UnZip{ index = 0.0964 value = 0.3208 dict = 1.00 all = 0.3041 } ---------------------------- total value len = 9.306689 GB avg = 0.166 KB (by entry num) total key len = 1.544923 GB avg = 0.028 KB total ukey len = 1.072236 GB avg = 0.015 KB total ukey num = 0.069268350 Billion total entry num = 0.056076079 Billion write speed all = 0.329075465 MB/sec (with seq num) write speed all = 0.315471394 MB/sec (without seq num) 2024/11/23-18:04:48.330114 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273847: 935481 keys, 40813252 bytes, temperature: kWarm 2024/11/23-18:04:48.330184 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356288330150, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273847, "file_size": 40813252, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4169436001, "table_properties": {"data_size": 34657712, "index_size": 1802944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26193468, "raw_average_key_size": 28, "raw_value_size": 108024322, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935481, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356284, "oldest_key_time": 0, "file_creation_time": 1732356284, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273847, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:04:48.369256 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273848.sst) /dev/shm/ToplingTemp/Topling-143155-qh2M8S 2024/11/23-18:04:49.240928 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273848.sst): first pass time = 0.87's, 153.984'MB/sec 2024/11/23-18:04:50.870517 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273848.sst): old prealloc_size = 147639500, real_size = 34764176 2024/11/23-18:04:50.871067 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273848.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927700 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0327 GB avg-val = 116.68 avg-zval = 35.30 2024/11/23-18:04:50.871744 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273848.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:04:51.861320 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273848.sst): second pass time = 1.63's, 66.472'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 38.842'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3365.467'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.270'MB/sec write SST data time = 0.13's, 296.384'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35164.111'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.472'MB/sec zip pipeline throughput = 66.472'MB/sec entries = 927700 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.68 avg-zval = 35.30 usrkeys = 927700 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.15 avg-zval = 35.30 TagRS{ kinds = RS_Key0_TagN bytes = 145040 } TagArray{ size = 724635 bytes = 2717392 } seq expand size = 5072445 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.3745 value = 3.3057 dict = 1.00 all = 3.4876 } Zip/UnZip{ index = 0.0964 value = 0.3025 dict = 1.00 all = 0.2867 } ---------------------------- total value len = 9.420003 GB avg = 0.165 KB (by entry num) total key len = 1.570898 GB avg = 0.028 KB total ukey len = 1.090790 GB avg = 0.016 KB total ukey num = 0.070196050 Billion total entry num = 0.057003779 Billion write speed all = 0.333263168 MB/sec (with seq num) write speed all = 0.319435542 MB/sec (without seq num) 2024/11/23-18:04:51.910771 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273848: 927700 keys, 38484376 bytes, temperature: kWarm 2024/11/23-18:04:51.910840 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356291910804, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273848, "file_size": 38484376, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775918, "table_properties": {"data_size": 32743680, "index_size": 1788416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975600, "raw_average_key_size": 28, "raw_value_size": 108242128, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927700, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356288, "oldest_key_time": 0, "file_creation_time": 1732356288, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273848, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:04:51.949890 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273849.sst) /dev/shm/ToplingTemp/Topling-143155-71K3wS 2024/11/23-18:04:52.966204 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273849.sst): first pass time = 1.02's, 132.071'MB/sec 2024/11/23-18:04:54.440703 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273849.sst): old prealloc_size = 147639500, real_size = 33172352 2024/11/23-18:04:54.441289 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273849.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929364 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.42 avg-zval = 33.54 2024/11/23-18:04:54.442119 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273849.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:04:55.579147 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273849.sst): second pass time = 1.47's, 73.438'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.14's, 32.949'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3124.655'MB/sec (index lex order gen) rebuild zvType time = 1.01's, 0.230'MB/sec write SST data time = 0.13's, 296.025'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34495.269'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.438'MB/sec zip pipeline throughput = 73.438'MB/sec entries = 929364 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.42 avg-zval = 33.54 usrkeys = 929364 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.42 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929364 bytes = 3485136 } seq expand size = 6505548 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5135 value = 3.4709 dict = 1.00 all = 3.5782 } Zip/UnZip{ index = 0.0951 value = 0.2881 dict = 1.00 all = 0.2795 } ---------------------------- total value len = 9.534704 GB avg = 0.165 KB (by entry num) total key len = 1.596920 GB avg = 0.028 KB total ukey len = 1.109378 GB avg = 0.016 KB total ukey num = 0.071125414 Billion total entry num = 0.057933143 Billion write speed all = 0.337492100 MB/sec (with seq num) write speed all = 0.323440618 MB/sec (without seq num) 2024/11/23-18:04:55.627802 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273849: 929364 keys, 37510004 bytes, temperature: kWarm 2024/11/23-18:04:55.627876 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356295627836, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273849, "file_size": 37510004, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012774578, "table_properties": {"data_size": 31171888, "index_size": 1767936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26022192, "raw_average_key_size": 28, "raw_value_size": 108195664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929364, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356291, "oldest_key_time": 0, "file_creation_time": 1732356291, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273849, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:04:55.663725 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273850.sst) /dev/shm/ToplingTemp/Topling-143155-7AlKKV 2024/11/23-18:04:56.738926 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273850.sst): first pass time = 1.08's, 124.835'MB/sec 2024/11/23-18:04:58.173788 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273850.sst): old prealloc_size = 147639500, real_size = 34817600 2024/11/23-18:04:58.174510 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273850.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928829 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0328 GB avg-val = 116.50 avg-zval = 35.32 2024/11/23-18:04:58.175397 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273850.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:04:59.165670 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273850.sst): second pass time = 1.43's, 75.462'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 39.358'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2535.460'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.277'MB/sec write SST data time = 0.15's, 257.658'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25153.681'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.462'MB/sec zip pipeline throughput = 75.462'MB/sec entries = 928829 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.50 avg-zval = 35.32 usrkeys = 928829 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 35.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928829 bytes = 3367024 } seq expand size = 6501803 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4451 value = 3.2984 dict = 1.00 all = 3.4382 } Zip/UnZip{ index = 0.0957 value = 0.3032 dict = 1.00 all = 0.2909 } ---------------------------- total value len = 9.649417 GB avg = 0.164 KB (by entry num) total key len = 1.622928 GB avg = 0.028 KB total ukey len = 1.127954 GB avg = 0.016 KB total ukey num = 0.072054243 Billion total entry num = 0.058861972 Billion write speed all = 0.341721324 MB/sec (with seq num) write speed all = 0.327446110 MB/sec (without seq num) 2024/11/23-18:04:59.217001 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273850: 928829 keys, 39037756 bytes, temperature: kWarm 2024/11/23-18:04:59.217082 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356299217036, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273850, "file_size": 39037756, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 32806704, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007212, "raw_average_key_size": 28, "raw_value_size": 108210614, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928829, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356295, "oldest_key_time": 0, "file_creation_time": 1732356295, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273850, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:04:59.258202 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273851.sst) /dev/shm/ToplingTemp/Topling-143155-DIFtRT 2024/11/23-18:05:00.366025 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273851.sst): first pass time = 1.11's, 121.160'MB/sec 2024/11/23-18:05:01.889053 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273851.sst): old prealloc_size = 147639500, real_size = 33556032 2024/11/23-18:05:01.889772 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273851.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927194 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.76 avg-zval = 34.04 2024/11/23-18:05:01.890681 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273851.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:05:02.864843 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273851.sst): second pass time = 1.52's, 71.122'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 38.710'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2534.970'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.15's, 244.085'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32954.774'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.122'MB/sec zip pipeline throughput = 71.122'MB/sec entries = 927194 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 34.04 usrkeys = 927194 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 34.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927194 bytes = 3361088 } seq expand size = 6490358 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5153 value = 3.4301 dict = 1.00 all = 3.5534 } Zip/UnZip{ index = 0.0951 value = 0.2915 dict = 1.00 all = 0.2814 } ---------------------------- total value len = 9.764164 GB avg = 0.163 KB (by entry num) total key len = 1.648889 GB avg = 0.028 KB total ukey len = 1.146498 GB avg = 0.016 KB total ukey num = 0.072981437 Billion total entry num = 0.059789166 Billion write speed all = 0.345948098 MB/sec (with seq num) write speed all = 0.331449647 MB/sec (without seq num) 2024/11/23-18:05:02.912254 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273851: 927194 keys, 37771836 bytes, temperature: kWarm 2024/11/23-18:05:02.912327 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356302912291, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273851, "file_size": 37771836, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4012776021, "table_properties": {"data_size": 31560560, "index_size": 1763520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961432, "raw_average_key_size": 28, "raw_value_size": 108256312, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927194, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356299, "oldest_key_time": 0, "file_creation_time": 1732356299, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273851, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:02.954539 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273852.sst) /dev/shm/ToplingTemp/Topling-143155-eFFvGV 2024/11/23-18:05:04.115549 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273852.sst): first pass time = 1.16's, 115.608'MB/sec 2024/11/23-18:05:05.688113 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273852.sst): old prealloc_size = 147639500, real_size = 35965136 2024/11/23-18:05:05.688695 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273852.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924278 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0340 GB avg-val = 117.21 avg-zval = 36.75 2024/11/23-18:05:05.689392 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273852.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:05:06.505313 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273852.sst): second pass time = 1.57's, 68.924'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 48.643'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3154.057'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.338'MB/sec write SST data time = 0.13's, 297.844'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34875.101'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.924'MB/sec zip pipeline throughput = 70.587'MB/sec entries = 924278 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.21 avg-zval = 36.75 usrkeys = 924278 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.83 avg-zval = 36.75 TagRS{ kinds = RS_Key0_TagN bytes = 144520 } TagArray{ size = 741281 bytes = 2779824 } seq expand size = 5188967 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.4450 value = 3.1898 dict = 1.00 all = 3.3766 } Zip/UnZip{ index = 0.0957 value = 0.3135 dict = 1.00 all = 0.2962 } ---------------------------- total value len = 9.877691 GB avg = 0.163 KB (by entry num) total key len = 1.674769 GB avg = 0.028 KB total ukey len = 1.164984 GB avg = 0.016 KB total ukey num = 0.073905715 Billion total entry num = 0.060713444 Billion write speed all = 0.350135106 MB/sec (with seq num) write speed all = 0.335414149 MB/sec (without seq num) 2024/11/23-18:05:06.556819 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273852: 924278 keys, 39749984 bytes, temperature: kWarm 2024/11/23-18:05:06.556893 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356306556856, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273852, "file_size": 39749984, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 33964096, "index_size": 1769792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25879784, "raw_average_key_size": 28, "raw_value_size": 108338032, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924278, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356302, "oldest_key_time": 0, "file_creation_time": 1732356302, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273852, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:06.595224 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273853.sst) /dev/shm/ToplingTemp/Topling-143155-5gsjhT 2024/11/23-18:05:07.672392 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273853.sst): first pass time = 1.08's, 124.608'MB/sec 2024/11/23-18:05:09.201752 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273853.sst): old prealloc_size = 147639500, real_size = 32628432 2024/11/23-18:05:09.202316 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273853.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932947 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.86 avg-zval = 32.82 2024/11/23-18:05:09.203124 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273853.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:05:10.069102 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273853.sst): second pass time = 1.53's, 70.733'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.499'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3269.416'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.13's, 285.709'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31944.642'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.733'MB/sec zip pipeline throughput = 70.733'MB/sec entries = 932947 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.86 avg-zval = 32.82 usrkeys = 932947 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.86 avg-zval = 32.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932947 bytes = 3381952 } seq expand size = 6530629 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5149 value = 3.5302 dict = 1.00 all = 3.6412 } Zip/UnZip{ index = 0.0951 value = 0.2833 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 9.992316 GB avg = 0.162 KB (by entry num) total key len = 1.700891 GB avg = 0.028 KB total ukey len = 1.183643 GB avg = 0.016 KB total ukey num = 0.074838662 Billion total entry num = 0.061646391 Billion write speed all = 0.354362673 MB/sec (with seq num) write speed all = 0.339417122 MB/sec (without seq num) 2024/11/23-18:05:10.115105 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273853: 932947 keys, 36860932 bytes, temperature: kWarm 2024/11/23-18:05:10.115174 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356310115141, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273853, "file_size": 36860932, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 30620480, "index_size": 1774528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26122516, "raw_average_key_size": 28, "raw_value_size": 108095306, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932947, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356306, "oldest_key_time": 0, "file_creation_time": 1732356306, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273853, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:10.154852 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273854.sst) /dev/shm/ToplingTemp/Topling-143155-Dlk5NW 2024/11/23-18:05:11.205263 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273854.sst): first pass time = 1.05's, 127.782'MB/sec 2024/11/23-18:05:12.804087 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273854.sst): old prealloc_size = 147639500, real_size = 32126848 2024/11/23-18:05:12.804670 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273854.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936222 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0301 GB avg-val = 115.36 avg-zval = 32.16 2024/11/23-18:05:12.805485 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273854.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:05:13.661657 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273854.sst): second pass time = 1.60's, 67.586'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.411'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3165.438'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 286.994'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 35952.918'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.586'MB/sec zip pipeline throughput = 69.876'MB/sec entries = 936222 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.36 avg-zval = 32.16 usrkeys = 936222 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.36 avg-zval = 32.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936222 bytes = 3393824 } seq expand size = 6553554 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5131 value = 3.5868 dict = 1.00 all = 3.6904 } Zip/UnZip{ index = 0.0951 value = 0.2788 dict = 1.00 all = 0.2710 } ---------------------------- total value len = 10.106874 GB avg = 0.161 KB (by entry num) total key len = 1.727106 GB avg = 0.028 KB total ukey len = 1.202367 GB avg = 0.016 KB total ukey num = 0.075774884 Billion total entry num = 0.062582613 Billion write speed all = 0.358589711 MB/sec (with seq num) write speed all = 0.343418833 MB/sec (without seq num) 2024/11/23-18:05:13.706945 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273854: 936222 keys, 36369214 bytes, temperature: kWarm 2024/11/23-18:05:13.707029 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356313706992, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273854, "file_size": 36369214, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30111536, "index_size": 1781056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26214216, "raw_average_key_size": 28, "raw_value_size": 108003648, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936222, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356310, "oldest_key_time": 0, "file_creation_time": 1732356310, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273854, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:13.743516 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273855.sst) /dev/shm/ToplingTemp/Topling-143155-C4fErT 2024/11/23-18:05:14.734396 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273855.sst): first pass time = 0.99's, 135.458'MB/sec 2024/11/23-18:05:16.105817 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273855.sst): old prealloc_size = 147639500, real_size = 34989904 2024/11/23-18:05:16.106379 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273855.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0330 GB avg-val = 117.01 avg-zval = 35.64 2024/11/23-18:05:16.107189 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273855.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:05:17.141696 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273855.sst): second pass time = 1.37's, 79.038'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 37.955'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3264.937'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.258'MB/sec write SST data time = 0.14's, 280.963'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32577.604'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.038'MB/sec zip pipeline throughput = 79.038'MB/sec entries = 925568 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.01 avg-zval = 35.64 usrkeys = 925568 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.01 avg-zval = 35.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925568 bytes = 3470896 } seq expand size = 6478976 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4434 value = 3.2833 dict = 1.00 all = 3.4138 } Zip/UnZip{ index = 0.0958 value = 0.3046 dict = 1.00 all = 0.2929 } ---------------------------- total value len = 10.221654 GB avg = 0.161 KB (by entry num) total key len = 1.753022 GB avg = 0.028 KB total ukey len = 1.220878 GB avg = 0.016 KB total ukey num = 0.076700452 Billion total entry num = 0.063508181 Billion write speed all = 0.362814801 MB/sec (with seq num) write speed all = 0.347421176 MB/sec (without seq num) 2024/11/23-18:05:17.192375 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273855: 925568 keys, 39316042 bytes, temperature: kWarm 2024/11/23-18:05:17.192446 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356317192402, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273855, "file_size": 39316042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 32985792, "index_size": 1772544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915904, "raw_average_key_size": 28, "raw_value_size": 108301824, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356313, "oldest_key_time": 0, "file_creation_time": 1732356313, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273855, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:17.232277 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273856.sst) /dev/shm/ToplingTemp/Topling-143155-OmdGOT 2024/11/23-18:05:18.376780 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273856.sst): first pass time = 1.14's, 117.275'MB/sec 2024/11/23-18:05:19.704586 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273856.sst): old prealloc_size = 147639500, real_size = 32186416 2024/11/23-18:05:19.705129 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273856.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0302 GB avg-val = 116.13 avg-zval = 32.41 2024/11/23-18:05:19.705918 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273856.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:05:20.548011 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273856.sst): second pass time = 1.33's, 81.494'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.175'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.629'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 287.419'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34449.527'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.494'MB/sec zip pipeline throughput = 81.494'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 32.41 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 32.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5116 value = 3.5831 dict = 1.00 all = 3.6859 } Zip/UnZip{ index = 0.0951 value = 0.2791 dict = 1.00 all = 0.2713 } ---------------------------- total value len = 10.336316 GB avg = 0.160 KB (by entry num) total key len = 1.779096 GB avg = 0.028 KB total ukey len = 1.239503 GB avg = 0.016 KB total ukey num = 0.077631694 Billion total entry num = 0.064439423 Billion write speed all = 0.367041024 MB/sec (with seq num) write speed all = 0.351423290 MB/sec (without seq num) 2024/11/23-18:05:20.593969 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273856: 931242 keys, 36413656 bytes, temperature: kWarm 2024/11/23-18:05:20.594045 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356320594010, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273856, "file_size": 36413656, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 30181600, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356317, "oldest_key_time": 0, "file_creation_time": 1732356317, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273856, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:20.635585 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273857.sst) /dev/shm/ToplingTemp/Topling-143155-2Del1T 2024/11/23-18:05:21.665921 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273857.sst): first pass time = 1.03's, 130.271'MB/sec 2024/11/23-18:05:23.035329 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273857.sst): old prealloc_size = 147639500, real_size = 35201056 2024/11/23-18:05:23.035849 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273857.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.46 avg-zval = 36.00 2024/11/23-18:05:23.036661 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273857.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:05:23.878196 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273857.sst): second pass time = 1.37's, 79.227'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.889'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3524.273'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 312.265'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35859.025'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.227'MB/sec zip pipeline throughput = 79.227'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 36.00 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 36.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5135 value = 3.2631 dict = 1.00 all = 3.3963 } Zip/UnZip{ index = 0.0951 value = 0.3065 dict = 1.00 all = 0.2944 } ---------------------------- total value len = 10.451157 GB avg = 0.160 KB (by entry num) total key len = 1.804932 GB avg = 0.028 KB total ukey len = 1.257957 GB avg = 0.016 KB total ukey num = 0.078554391 Billion total entry num = 0.065362120 Billion write speed all = 0.371265421 MB/sec (with seq num) write speed all = 0.355425657 MB/sec (without seq num) 2024/11/23-18:05:23.926280 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273857: 922697 keys, 39519390 bytes, temperature: kWarm 2024/11/23-18:05:23.926353 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356323926316, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273857, "file_size": 39519390, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33214928, "index_size": 1755264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356320, "oldest_key_time": 0, "file_creation_time": 1732356320, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273857, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:23.970202 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273858.sst) /dev/shm/ToplingTemp/Topling-143155-JXZnJS 2024/11/23-18:05:24.876006 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273858.sst): first pass time = 0.91's, 148.182'MB/sec 2024/11/23-18:05:26.183832 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273858.sst): old prealloc_size = 147639500, real_size = 32818464 2024/11/23-18:05:26.184407 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273858.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.15 avg-zval = 33.08 2024/11/23-18:05:26.185244 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273858.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:05:27.044956 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273858.sst): second pass time = 1.31's, 82.740'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.156'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3222.393'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 291.198'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31774.183'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.740'MB/sec zip pipeline throughput = 82.740'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.15 avg-zval = 33.08 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.15 avg-zval = 33.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4438 value = 3.5110 dict = 1.00 all = 3.6117 } Zip/UnZip{ index = 0.0958 value = 0.2848 dict = 1.00 all = 0.2769 } ---------------------------- total value len = 10.565822 GB avg = 0.159 KB (by entry num) total key len = 1.831002 GB avg = 0.028 KB total ukey len = 1.276579 GB avg = 0.016 KB total ukey num = 0.079485474 Billion total entry num = 0.066293203 Billion write speed all = 0.375492603 MB/sec (with seq num) write speed all = 0.359428743 MB/sec (without seq num) 2024/11/23-18:05:27.089707 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273858: 931083 keys, 37161536 bytes, temperature: kWarm 2024/11/23-18:05:27.089779 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356327089742, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273858, "file_size": 37161536, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30802448, "index_size": 1783040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356323, "oldest_key_time": 0, "file_creation_time": 1732356323, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273858, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:27.138712 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273859.sst) /dev/shm/ToplingTemp/Topling-143155-StDxPT 2024/11/23-18:05:28.127779 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273859.sst): first pass time = 0.99's, 135.707'MB/sec 2024/11/23-18:05:29.504080 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273859.sst): old prealloc_size = 147639500, real_size = 34036432 2024/11/23-18:05:29.504620 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273859.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927688 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0320 GB avg-val = 116.68 avg-zval = 34.52 2024/11/23-18:05:29.505204 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273859.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:05:30.307135 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273859.sst): second pass time = 1.38's, 78.716'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 46.255'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3419.222'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.343'MB/sec write SST data time = 0.13's, 294.519'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33035.748'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.716'MB/sec zip pipeline throughput = 78.716'MB/sec entries = 927688 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.68 avg-zval = 34.52 usrkeys = 927688 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.22 avg-zval = 34.52 TagRS{ kinds = RS_Key0_TagN bytes = 145040 } TagArray{ size = 601145 bytes = 2104016 } seq expand size = 4208015 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4424 value = 3.3797 dict = 1.00 all = 3.6134 } Zip/UnZip{ index = 0.0958 value = 0.2959 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 10.678273 GB avg = 0.159 KB (by entry num) total key len = 1.856977 GB avg = 0.028 KB total ukey len = 1.295133 GB avg = 0.016 KB total ukey num = 0.080413162 Billion total entry num = 0.067220891 Billion write speed all = 0.379647927 MB/sec (with seq num) write speed all = 0.363360883 MB/sec (without seq num) 2024/11/23-18:05:30.353245 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273859: 927688 keys, 37144446 bytes, temperature: kWarm 2024/11/23-18:05:30.353319 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356330353281, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273859, "file_size": 37144446, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 32027584, "index_size": 1776768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975264, "raw_average_key_size": 28, "raw_value_size": 108242516, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927688, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356327, "oldest_key_time": 0, "file_creation_time": 1732356327, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273859, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:30.392783 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273860.sst) /dev/shm/ToplingTemp/Topling-143155-mXFhbS 2024/11/23-18:05:31.390660 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273860.sst): first pass time = 1.00's, 134.508'MB/sec 2024/11/23-18:05:32.886170 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273860.sst): old prealloc_size = 147639500, real_size = 36853424 2024/11/23-18:05:32.886713 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273860.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926690 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.84 avg-zval = 37.62 2024/11/23-18:05:32.886935 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273860.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:05:33.603430 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273860.sst): second pass time = 1.49's, 72.438'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 53.453'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3483.778'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.391'MB/sec write SST data time = 0.12's, 309.709'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33504.481'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.438'MB/sec zip pipeline throughput = 72.438'MB/sec entries = 926690 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.84 avg-zval = 37.62 usrkeys = 926690 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.00 avg-zval = 37.62 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5137 value = 3.1060 dict = 1.00 all = 3.5008 } Zip/UnZip{ index = 0.0951 value = 0.3220 dict = 1.00 all = 0.2856 } ---------------------------- total value len = 10.787624 GB avg = 0.158 KB (by entry num) total key len = 1.882925 GB avg = 0.028 KB total ukey len = 1.313666 GB avg = 0.016 KB total ukey num = 0.081339852 Billion total entry num = 0.068147581 Billion write speed all = 0.383707330 MB/sec (with seq num) write speed all = 0.367197405 MB/sec (without seq num) 2024/11/23-18:05:33.650059 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273860: 926690 keys, 38339189 bytes, temperature: kWarm 2024/11/23-18:05:33.650133 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356333650094, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273860, "file_size": 38339189, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34858720, "index_size": 1762816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947320, "raw_average_key_size": 28, "raw_value_size": 108270440, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926690, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356330, "oldest_key_time": 0, "file_creation_time": 1732356330, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273860, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:33.687120 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273861.sst) /dev/shm/ToplingTemp/Topling-143155-J4uMwW 2024/11/23-18:05:34.730416 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273861.sst): first pass time = 1.04's, 128.653'MB/sec 2024/11/23-18:05:36.075953 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273861.sst): old prealloc_size = 147639500, real_size = 35810192 2024/11/23-18:05:36.076507 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926983 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0338 GB avg-val = 116.79 avg-zval = 36.49 2024/11/23-18:05:36.077154 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273861.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:05:36.884355 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273861.sst): second pass time = 1.34's, 80.506'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 48.675'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3409.684'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.341'MB/sec write SST data time = 0.13's, 304.747'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32870.261'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.506'MB/sec zip pipeline throughput = 80.506'MB/sec entries = 926983 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.79 avg-zval = 36.49 usrkeys = 926983 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.26 avg-zval = 36.49 TagRS{ kinds = RS_Key0_TagN bytes = 144960 } TagArray{ size = 724225 bytes = 2534800 } seq expand size = 5069575 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5843 value = 3.2005 dict = 1.00 all = 3.4110 } Zip/UnZip{ index = 0.0945 value = 0.3125 dict = 1.00 all = 0.2932 } ---------------------------- total value len = 10.900956 GB avg = 0.158 KB (by entry num) total key len = 1.908880 GB avg = 0.028 KB total ukey len = 1.332206 GB avg = 0.016 KB total ukey num = 0.082266835 Billion total entry num = 0.069074564 Billion write speed all = 0.387886886 MB/sec (with seq num) write speed all = 0.371154046 MB/sec (without seq num) 2024/11/23-18:05:36.931896 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273861: 926983 keys, 39348042 bytes, temperature: kWarm 2024/11/23-18:05:36.931966 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356336931931, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273861, "file_size": 39348042, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4169436350, "table_properties": {"data_size": 33826624, "index_size": 1751616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955524, "raw_average_key_size": 28, "raw_value_size": 108262286, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926983, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356333, "oldest_key_time": 0, "file_creation_time": 1732356333, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:36.967597 139957374625344 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273862.sst) /dev/shm/ToplingTemp/Topling-143155-VHRAHT 2024/11/23-18:05:37.055823 139957374625344 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273862.sst): first pass time = 0.09's, 214.316'MB/sec 2024/11/23-18:05:37.233572 139957374625344 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273862.sst): old prealloc_size = 147639500, real_size = 5387344 2024/11/23-18:05:37.233710 139957374625344 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 132182 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0152 GB zip-val = 0.0051 GB avg-val = 114.99 avg-zval = 38.59 2024/11/23-18:05:37.233870 139957374625344 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273862.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.019 GB 2024/11/23-18:05:37.349731 139957374625344 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00/273862.sst): second pass time = 0.18's, 86.186'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 51.321'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2377.480'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.340'MB/sec write SST data time = 0.02's, 316.946'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.158'MB, throughput = 4785.399'MB/sec zip my value time = 0.18's, unzip length = 0.015'GB zip my value throughput = 86.186'MB/sec zip pipeline throughput = 86.186'MB/sec entries = 132182 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.99 avg-zval = 38.59 usrkeys = 132182 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.99 avg-zval = 38.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 132182 bytes = 446128 } seq expand size = 925274 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0026 GB value = 0.0152 GB dict = 0.16 MB all = 0.0189 GB } __ZipSize{ index = 0.0003 GB value = 0.0051 GB dict = 0.16 MB all = 0.0060 GB } UnZip/Zip{ index = 10.4548 value = 2.9795 dict = 1.00 all = 3.1708 } Zip/UnZip{ index = 0.0956 value = 0.3356 dict = 1.00 all = 0.3154 } ---------------------------- total value len = 10.917080 GB avg = 0.158 KB (by entry num) total key len = 1.912581 GB avg = 0.028 KB total ukey len = 1.334850 GB avg = 0.016 KB total ukey num = 0.082399017 Billion total entry num = 0.069206746 Billion write speed all = 0.388481734 MB/sec (with seq num) write speed all = 0.371717110 MB/sec (without seq num) 2024/11/23-18:05:37.363223 139957374625344 [compaction_job.cc:1948] [default] [JOB 32639] Generated table #273862: 132182 keys, 5960623 bytes, temperature: kWarm 2024/11/23-18:05:37.363289 139957374625344 EVENT_LOG_v1 {"time_micros": 1732356337363252, "cf_name": "default", "job": 32639, "event": "table_file_creation", "file_number": 273862, "file_size": 5960623, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4169436164, "table_properties": {"data_size": 5101248, "index_size": 252864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3701096, "raw_average_key_size": 28, "raw_value_size": 15199052, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 132182, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356336, "oldest_key_time": 0, "file_creation_time": 1732356336, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:05:37.374672 139957374625344 [compaction_job.cc:703] job-32639: subcompact[0], size: 0.697430 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32639/att-00 [273844,273845,273846,273847,273848,273849,273850,273851,273852,273853,273854,273855,273856,273857,273858,273859,273860,273861,273862] 2024/11/23-18:05:37.516544 139957374625344 (Original Log Time 2024/11/23-18:04:28.756612) EVENT_LOG_v1 {"time_micros": 1732356268756557, "job": 32639, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [273837], "files_L3": [273752, 273753, 273792, 273793, 273794, 273795, 273796, 273797, 273798, 273799, 273800, 273801, 273802, 273803, 273804, 273805, 273806, 273807], "score": 0.99511, "input_data_size": 716804754, "oldest_snapshot_seqno": 4169464215} 2024/11/23-18:05:37.516548 139957374625344 (Original Log Time 2024/11/23-18:05:37.515012) [compaction_job.cc:2022] [default] [JOB 32639] Compacted 1@2 + 18@3 files to L3 => 697429588 bytes 2024/11/23-18:05:37.516550 139957374625344 (Original Log Time 2024/11/23-18:05:37.516478) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 19 0 0 0] max score 0.03, MB/sec: 10.4 rd, 10.2 wr, level 3, files in(1, 18) out(19 +0 blob) MB in(25.7, 657.9 +0.0 blob) out(665.1 +0.0 blob), read-write-amplify(52.5) write-amplify(25.9) OK, records in: 16939087, records dropped: 90505 output_compression: Snappy 2024/11/23-18:05:37.516555 139957374625344 (Original Log Time 2024/11/23-18:05:37.516523) EVENT_LOG_v1 {"time_micros": 1732356337516502, "cf": "default", "job": 32639, "event": "compaction_finished", "compaction_time_micros": 68618066, "compaction_time_cpu_micros": 28641128, "output_level": 3, "num_output_files": 19, "total_output_size": 697429588, "num_input_records": 16939087, "num_output_records": 16848582, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 558283800, "file_range_sync_nanos": 0, "file_fsync_nanos": 14093, "file_prepare_write_nanos": 50720, "lsm_state": [0, 0, 0, 19, 0, 0, 0]} 2024/11/23-18:05:37.655259 139957374625344 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732356268,"executesMs":68925,"compactionJobId":32639,"attempt":0,"compactionInputRawBytes":2447672166,"compactionInputZipBytes":716804754,"compactionOutputRawBytes":0,"compactionOutputZipBytes":697429588,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-5f287d3f3da3f7372749c0ab2b5fad02-3dadb8769740d417-00"}}, response = 2024/11/23-18:05:37.655341 139957374625344 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32639/att-00: olev 3, work 68.926 s, result 6.053 ms, install 1.561 ms, input{raw 2.280 GiB zip 683.598 MiB}