2024/11/24-04:42:14.736391 139957257127488 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00: fork to child time = 0.007689 sec 2024/11/24-04:42:14.737249 139957257127488 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00 2024/11/24-04:42:14.742797 139957257127488 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00 2024/11/24-04:42:14.743727 139957257127488 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-04:42:14.743968 139957257127488 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-04:42:14.795567 139957257127488 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41702/att-00/MANIFEST-206347 2024/11/24-04:42:14.812304 139957257127488 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41702/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 292347, last_sequence is 4444200838, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 292178 2024/11/24-04:42:14.812310 139957257127488 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-04:42:14.812439 139957257127488 [version_set.cc:5721] Creating manifest 292347 2024/11/24-04:42:14.847019 139957257127488 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-04:42:14.865717 139957257127488 [compaction_job.cc:2437] [default] [JOB 41702] Compacting 1@2 + 23@3 files to L3, score 0.83, subcompactions 1 : 1 2024/11/24-04:42:14.865732 139957257127488 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31257 Base level 2, inputs: [292323(27MB)], [292278(34MB) 292279(33MB) 292280(35MB) 292281(35MB) 292282(32MB) 292283(30MB) 292284(35MB) 292285(32MB) 292286(34MB) 292287(33MB) 292288(33MB) 292289(33MB) 292290(33MB) 292291(6691KB) 292225(33MB) 292235(34MB) 292236(33MB) 292237(33MB) 292238(33MB) 292239(33MB) 292240(33MB) 292241(32MB) 292341(34MB)] 2024/11/24-04:42:14.872664 139957257127488 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:42:14.872693 139957257127488 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:42:14.872712 139957257127488 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-04:42:14.872717 139957257127488 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:42:14.872727 139957257127488 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-04:42:15.461095 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292348.sst) /dev/shm/ToplingTemp/Topling-272325-uTO747 2024/11/24-04:42:16.031362 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292348.sst): first pass time = 0.57's, 235.393'MB/sec 2024/11/24-04:42:17.655490 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292348.sst): old prealloc_size = 147639500, real_size = 31752784 2024/11/24-04:42:17.656194 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292348.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927213 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 116.75 avg-zval = 32.08 2024/11/24-04:42:17.657081 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292348.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:18.816322 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292348.sst): second pass time = 1.62's, 66.702'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.16's, 30.885'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2569.537'MB/sec (index lex order gen) rebuild zvType time = 1.01's, 0.230'MB/sec write SST data time = 0.15's, 235.650'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33297.504'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.702'MB/sec zip pipeline throughput = 66.702'MB/sec entries = 927213 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.75 avg-zval = 32.08 usrkeys = 927213 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.75 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927213 bytes = 3245264 } seq expand size = 6490491 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.4457 value = 3.6394 dict = 1.00 all = 3.7436 } Zip/UnZip{ index = 0.0957 value = 0.2748 dict = 1.00 all = 0.2671 } ---------------------------- total value len = 2251.440320 GB avg = 0.200 KB (by entry num) total key len = 306.908924 GB avg = 0.027 KB total ukey len = 237.373272 GB avg = 0.015 KB total ukey num = 15.739588376 Billion total entry num = 11.280902324 Billion write speed all = 35.918451585 MB/sec (with seq num) write speed all = 34.651407862 MB/sec (without seq num) 2024/11/24-04:42:18.861442 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292348: 927213 keys, 35852424 bytes, temperature: kWarm 2024/11/24-04:42:18.861532 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394538861477, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292348, "file_size": 35852424, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4444082388, "table_properties": {"data_size": 29745472, "index_size": 1775296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961964, "raw_average_key_size": 28, "raw_value_size": 108255866, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927213, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394535, "oldest_key_time": 0, "file_creation_time": 1732394535, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292348, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:18.896722 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292349.sst) /dev/shm/ToplingTemp/Topling-272325-Bl0rI9 2024/11/24-04:42:19.903075 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292349.sst): first pass time = 1.01's, 133.375'MB/sec 2024/11/24-04:42:21.358992 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292349.sst): old prealloc_size = 147639500, real_size = 31477424 2024/11/24-04:42:21.359529 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292349.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.08 2024/11/24-04:42:21.360298 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292349.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:22.178846 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292349.sst): second pass time = 1.45's, 74.576'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.367'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3385.954'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 278.638'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33697.009'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.576'MB/sec zip pipeline throughput = 74.576'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.08 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4447 value = 3.6788 dict = 1.00 all = 3.7750 } Zip/UnZip{ index = 0.0957 value = 0.2718 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2251.555232 GB avg = 0.200 KB (by entry num) total key len = 306.934665 GB avg = 0.027 KB total ukey len = 237.391658 GB avg = 0.015 KB total ukey num = 15.740507676 Billion total entry num = 11.281821624 Billion write speed all = 35.918730633 MB/sec (with seq num) write speed all = 34.651643474 MB/sec (without seq num) 2024/11/24-04:42:22.226517 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292349: 919300 keys, 35554162 bytes, temperature: kWarm 2024/11/24-04:42:22.226589 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394542226553, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292349, "file_size": 35554162, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4281898976, "largest_seqno": 4434139959, "table_properties": {"data_size": 29487072, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394538, "oldest_key_time": 0, "file_creation_time": 1732394538, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292349, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:22.270145 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292350.sst) /dev/shm/ToplingTemp/Topling-272325-YmDLoa 2024/11/24-04:42:23.117119 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292350.sst): first pass time = 0.85's, 158.475'MB/sec 2024/11/24-04:42:24.535236 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292350.sst): old prealloc_size = 147639500, real_size = 33315104 2024/11/24-04:42:24.535903 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292350.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926862 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.81 avg-zval = 33.79 2024/11/24-04:42:24.536775 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292350.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:25.476269 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292350.sst): second pass time = 1.42's, 76.438'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.758'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2734.311'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.15's, 243.854'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33632.712'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.438'MB/sec zip pipeline throughput = 76.438'MB/sec entries = 926862 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 33.79 usrkeys = 926862 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.81 avg-zval = 33.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926862 bytes = 3244032 } seq expand size = 6488034 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5149 value = 3.4567 dict = 1.00 all = 3.5874 } Zip/UnZip{ index = 0.0951 value = 0.2893 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 2251.669986 GB avg = 0.200 KB (by entry num) total key len = 306.960617 GB avg = 0.027 KB total ukey len = 237.410195 GB avg = 0.015 KB total ukey num = 15.741434538 Billion total entry num = 11.282748486 Billion write speed all = 35.919043226 MB/sec (with seq num) write speed all = 34.651910627 MB/sec (without seq num) 2024/11/24-04:42:25.524528 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292350: 926862 keys, 37413244 bytes, temperature: kWarm 2024/11/24-04:42:25.524604 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394545524564, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292350, "file_size": 37413244, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4444082180, "table_properties": {"data_size": 31320272, "index_size": 1762944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952136, "raw_average_key_size": 28, "raw_value_size": 108265656, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926862, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394542, "oldest_key_time": 0, "file_creation_time": 1732394542, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292350, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:25.562644 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292351.sst) /dev/shm/ToplingTemp/Topling-272325-UYRcd8 2024/11/24-04:42:26.621164 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292351.sst): first pass time = 1.06's, 126.802'MB/sec 2024/11/24-04:42:28.073754 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292351.sst): old prealloc_size = 147639500, real_size = 32433536 2024/11/24-04:42:28.074333 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292351.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927306 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 116.74 avg-zval = 32.81 2024/11/24-04:42:28.075146 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292351.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:28.904493 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292351.sst): second pass time = 1.45's, 74.569'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.983'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3188.651'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.12's, 293.666'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30812.058'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.569'MB/sec zip pipeline throughput = 74.569'MB/sec entries = 927306 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.74 avg-zval = 32.81 usrkeys = 927306 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.74 avg-zval = 32.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927306 bytes = 3245584 } seq expand size = 6491142 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4449 value = 3.5579 dict = 1.00 all = 3.6735 } Zip/UnZip{ index = 0.0957 value = 0.2811 dict = 1.00 all = 0.2722 } ---------------------------- total value len = 2251.784730 GB avg = 0.200 KB (by entry num) total key len = 306.986582 GB avg = 0.027 KB total ukey len = 237.428741 GB avg = 0.015 KB total ukey num = 15.742361844 Billion total entry num = 11.283675792 Billion write speed all = 35.919289870 MB/sec (with seq num) write speed all = 34.652114113 MB/sec (without seq num) 2024/11/24-04:42:28.950206 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292351: 927306 keys, 36537050 bytes, temperature: kWarm 2024/11/24-04:42:28.950287 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394548950251, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292351, "file_size": 36537050, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4444082194, "table_properties": {"data_size": 30425904, "index_size": 1775616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25964568, "raw_average_key_size": 28, "raw_value_size": 108253184, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927306, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394545, "oldest_key_time": 0, "file_creation_time": 1732394545, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292351, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:28.988148 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292352.sst) /dev/shm/ToplingTemp/Topling-272325-tQj6P8 2024/11/24-04:42:29.856953 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292352.sst): first pass time = 0.87's, 154.492'MB/sec 2024/11/24-04:42:31.293657 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292352.sst): old prealloc_size = 147639500, real_size = 30198016 2024/11/24-04:42:31.294387 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292352.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930699 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0282 GB avg-val = 116.21 avg-zval = 30.28 2024/11/24-04:42:31.295255 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292352.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:32.122409 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292352.sst): second pass time = 1.44's, 75.370'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.394'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2518.488'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.14's, 251.991'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31434.676'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.370'MB/sec zip pipeline throughput = 75.370'MB/sec entries = 930699 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.21 avg-zval = 30.28 usrkeys = 930699 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.21 avg-zval = 30.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930699 bytes = 3257456 } seq expand size = 6514893 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0282 GB dict = 1.08 MB all = 0.0343 GB } UnZip/Zip{ index = 10.4443 value = 3.8377 dict = 1.00 all = 3.9125 } Zip/UnZip{ index = 0.0957 value = 0.2606 dict = 1.00 all = 0.2556 } ---------------------------- total value len = 2251.899403 GB avg = 0.200 KB (by entry num) total key len = 307.012641 GB avg = 0.027 KB total ukey len = 237.447355 GB avg = 0.015 KB total ukey num = 15.743292543 Billion total entry num = 11.284606491 Billion write speed all = 35.919642866 MB/sec (with seq num) write speed all = 34.652419834 MB/sec (without seq num) 2024/11/24-04:42:32.165302 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292352: 930699 keys, 34305166 bytes, temperature: kWarm 2024/11/24-04:42:32.165372 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394552165337, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292352, "file_size": 34305166, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4444081858, "table_properties": {"data_size": 28182960, "index_size": 1782208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26059572, "raw_average_key_size": 28, "raw_value_size": 108158174, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930699, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394548, "oldest_key_time": 0, "file_creation_time": 1732394548, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292352, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:32.201160 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292353.sst) /dev/shm/ToplingTemp/Topling-272325-zWLEG9 2024/11/24-04:42:33.177597 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292353.sst): first pass time = 0.98's, 137.462'MB/sec 2024/11/24-04:42:34.440318 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292353.sst): old prealloc_size = 147639500, real_size = 27848976 2024/11/24-04:42:34.440918 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292353.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0258 GB avg-val = 114.00 avg-zval = 27.30 2024/11/24-04:42:34.441710 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292353.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:35.276814 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292353.sst): second pass time = 1.26's, 85.387'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 38.113'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3149.237'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.333'MB/sec write SST data time = 0.13's, 251.868'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 35502.696'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.387'MB/sec zip pipeline throughput = 85.387'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.30 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0258 GB dict = 1.07 MB all = 0.0319 GB } UnZip/Zip{ index = 10.4457 value = 4.1760 dict = 1.00 all = 4.2101 } Zip/UnZip{ index = 0.0957 value = 0.2395 dict = 1.00 all = 0.2375 } ---------------------------- total value len = 2252.013772 GB avg = 0.200 KB (by entry num) total key len = 307.039107 GB avg = 0.027 KB total ukey len = 237.466259 GB avg = 0.015 KB total ukey num = 15.744237739 Billion total entry num = 11.285551687 Billion write speed all = 35.920029276 MB/sec (with seq num) write speed all = 34.652756214 MB/sec (without seq num) 2024/11/24-04:42:35.319006 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292353: 945196 keys, 31880176 bytes, temperature: kWarm 2024/11/24-04:42:35.319080 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394555319044, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292353, "file_size": 31880176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4309121445, "largest_seqno": 4410573408, "table_properties": {"data_size": 25802752, "index_size": 1809728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394552, "oldest_key_time": 0, "file_creation_time": 1732394552, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292353, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:35.355909 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292354.sst) /dev/shm/ToplingTemp/Topling-272325-LvViI7 2024/11/24-04:42:36.382492 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292354.sst): first pass time = 1.03's, 130.747'MB/sec 2024/11/24-04:42:37.710644 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292354.sst): old prealloc_size = 147639500, real_size = 33088000 2024/11/24-04:42:37.711230 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292354.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927626 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.69 avg-zval = 33.52 2024/11/24-04:42:37.712026 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292354.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:38.566610 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292354.sst): second pass time = 1.33's, 81.547'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.448'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3136.312'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 294.337'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32572.282'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.547'MB/sec zip pipeline throughput = 81.547'MB/sec entries = 927626 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.69 avg-zval = 33.52 usrkeys = 927626 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.69 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927626 bytes = 3246704 } seq expand size = 6493382 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5118 value = 3.4815 dict = 1.00 all = 3.6091 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 2252.128509 GB avg = 0.200 KB (by entry num) total key len = 307.065080 GB avg = 0.027 KB total ukey len = 237.484811 GB avg = 0.015 KB total ukey num = 15.745165365 Billion total entry num = 11.286479313 Billion write speed all = 35.920345669 MB/sec (with seq num) write speed all = 34.653026964 MB/sec (without seq num) 2024/11/24-04:42:38.612736 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292354: 927626 keys, 37188558 bytes, temperature: kWarm 2024/11/24-04:42:38.612808 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394558612772, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292354, "file_size": 37188558, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4444080454, "table_properties": {"data_size": 31090992, "index_size": 1764928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25973528, "raw_average_key_size": 28, "raw_value_size": 108244200, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927626, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394555, "oldest_key_time": 0, "file_creation_time": 1732394555, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292354, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:38.649252 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292355.sst) /dev/shm/ToplingTemp/Topling-272325-HR1NR9 2024/11/24-04:42:39.619557 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292355.sst): first pass time = 0.97's, 138.331'MB/sec 2024/11/24-04:42:40.923610 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292355.sst): old prealloc_size = 147639500, real_size = 30109824 2024/11/24-04:42:40.924166 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292355.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930805 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0281 GB avg-val = 116.20 avg-zval = 30.17 2024/11/24-04:42:40.924933 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292355.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:41.733344 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292355.sst): second pass time = 1.30's, 82.988'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.115'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3365.697'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 271.244'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 23346.307'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 82.988'MB/sec zip pipeline throughput = 82.988'MB/sec entries = 930805 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.20 avg-zval = 30.17 usrkeys = 930805 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.20 avg-zval = 30.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930805 bytes = 3141488 } seq expand size = 6515635 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0281 GB dict = 1.08 MB all = 0.0341 GB } UnZip/Zip{ index = 10.3770 value = 3.8513 dict = 1.00 all = 3.9359 } Zip/UnZip{ index = 0.0964 value = 0.2597 dict = 1.00 all = 0.2541 } ---------------------------- total value len = 2252.243180 GB avg = 0.200 KB (by entry num) total key len = 307.091143 GB avg = 0.027 KB total ukey len = 237.503427 GB avg = 0.015 KB total ukey num = 15.746096170 Billion total entry num = 11.287410118 Billion write speed all = 35.920724382 MB/sec (with seq num) write speed all = 34.653357492 MB/sec (without seq num) 2024/11/24-04:42:41.778360 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292355: 930805 keys, 34100986 bytes, temperature: kWarm 2024/11/24-04:42:41.778432 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394561778396, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292355, "file_size": 34100986, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4327142152, "largest_seqno": 4444081874, "table_properties": {"data_size": 28082928, "index_size": 1793984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26062540, "raw_average_key_size": 28, "raw_value_size": 108155326, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930805, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394558, "oldest_key_time": 0, "file_creation_time": 1732394558, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292355, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:41.815056 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292356.sst) /dev/shm/ToplingTemp/Topling-272325-0AVbn7 2024/11/24-04:42:42.673702 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292356.sst): first pass time = 0.86's, 156.322'MB/sec 2024/11/24-04:42:43.984841 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292356.sst): old prealloc_size = 147639500, real_size = 31515648 2024/11/24-04:42:43.985408 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292356.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 937310 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0295 GB avg-val = 115.19 avg-zval = 31.47 2024/11/24-04:42:43.986197 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292356.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:44.834892 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292356.sst): second pass time = 1.31's, 82.398'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.933'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3276.858'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 282.448'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 36341.480'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.398'MB/sec zip pipeline throughput = 82.398'MB/sec entries = 937310 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.19 avg-zval = 31.47 usrkeys = 937310 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.19 avg-zval = 31.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 937310 bytes = 3280592 } seq expand size = 6561170 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5136 value = 3.6603 dict = 1.00 all = 3.7655 } Zip/UnZip{ index = 0.0951 value = 0.2732 dict = 1.00 all = 0.2656 } ---------------------------- total value len = 2252.357715 GB avg = 0.200 KB (by entry num) total key len = 307.117387 GB avg = 0.027 KB total ukey len = 237.522174 GB avg = 0.015 KB total ukey num = 15.747033480 Billion total entry num = 11.288347428 Billion write speed all = 35.921136563 MB/sec (with seq num) write speed all = 34.653719602 MB/sec (without seq num) 2024/11/24-04:42:44.877900 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292356: 937310 keys, 35644404 bytes, temperature: kWarm 2024/11/24-04:42:44.877971 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394564877935, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292356, "file_size": 35644404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4444082519, "table_properties": {"data_size": 29498096, "index_size": 1783040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26244680, "raw_average_key_size": 28, "raw_value_size": 107973052, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 937310, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394561, "oldest_key_time": 0, "file_creation_time": 1732394561, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292356, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:44.920873 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292357.sst) /dev/shm/ToplingTemp/Topling-272325-gyV5M7 2024/11/24-04:42:45.802917 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292357.sst): first pass time = 0.88's, 152.173'MB/sec 2024/11/24-04:42:47.084156 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292357.sst): old prealloc_size = 147639500, real_size = 31329600 2024/11/24-04:42:47.084688 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292357.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.91 2024/11/24-04:42:47.085461 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292357.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:47.894297 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292357.sst): second pass time = 1.28's, 84.724'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.563'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3477.372'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 281.767'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30742.924'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.724'MB/sec zip pipeline throughput = 84.724'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.91 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4451 value = 3.6973 dict = 1.00 all = 3.8031 } Zip/UnZip{ index = 0.0957 value = 0.2705 dict = 1.00 all = 0.2629 } ---------------------------- total value len = 2252.472627 GB avg = 0.200 KB (by entry num) total key len = 307.143128 GB avg = 0.027 KB total ukey len = 237.540560 GB avg = 0.015 KB total ukey num = 15.747952780 Billion total entry num = 11.289266728 Billion write speed all = 35.921568185 MB/sec (with seq num) write speed all = 34.654102430 MB/sec (without seq num) 2024/11/24-04:42:47.937018 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292357: 919300 keys, 35291442 bytes, temperature: kWarm 2024/11/24-04:42:47.937091 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394567937055, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292357, "file_size": 35291442, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4299583081, "largest_seqno": 4418016673, "table_properties": {"data_size": 29339312, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394564, "oldest_key_time": 0, "file_creation_time": 1732394564, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292357, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:47.976991 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292358.sst) /dev/shm/ToplingTemp/Topling-272325-vPhGUa 2024/11/24-04:42:48.987046 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292358.sst): first pass time = 1.01's, 132.888'MB/sec 2024/11/24-04:42:50.303491 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292358.sst): old prealloc_size = 147639500, real_size = 30937936 2024/11/24-04:42:50.304054 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292358.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935914 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0289 GB avg-val = 115.41 avg-zval = 30.90 2024/11/24-04:42:50.304862 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292358.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:51.139239 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292358.sst): second pass time = 1.32's, 82.121'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 41.956'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3302.814'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.330'MB/sec write SST data time = 0.13's, 278.883'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34319.210'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.121'MB/sec zip pipeline throughput = 82.121'MB/sec entries = 935914 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.41 avg-zval = 30.90 usrkeys = 935914 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.41 avg-zval = 30.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935914 bytes = 3275712 } seq expand size = 6551398 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5108 value = 3.7345 dict = 1.00 all = 3.8279 } Zip/UnZip{ index = 0.0951 value = 0.2678 dict = 1.00 all = 0.2612 } ---------------------------- total value len = 2252.587191 GB avg = 0.200 KB (by entry num) total key len = 307.169333 GB avg = 0.027 KB total ukey len = 237.559278 GB avg = 0.015 KB total ukey num = 15.748888694 Billion total entry num = 11.290202642 Billion write speed all = 35.921907879 MB/sec (with seq num) write speed all = 34.654394769 MB/sec (without seq num) 2024/11/24-04:42:51.182124 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292358: 935914 keys, 35062832 bytes, temperature: kWarm 2024/11/24-04:42:51.182200 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394571182161, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292358, "file_size": 35062832, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4444082766, "table_properties": {"data_size": 28922880, "index_size": 1780864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26205592, "raw_average_key_size": 28, "raw_value_size": 108012280, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935914, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394567, "oldest_key_time": 0, "file_creation_time": 1732394567, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292358, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:51.224102 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292359.sst) /dev/shm/ToplingTemp/Topling-272325-OpIl26 2024/11/24-04:42:52.227825 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292359.sst): first pass time = 1.00's, 133.725'MB/sec 2024/11/24-04:42:53.521559 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292359.sst): old prealloc_size = 147639500, real_size = 31304704 2024/11/24-04:42:53.522116 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292359.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.89 2024/11/24-04:42:53.522875 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292359.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:54.332627 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292359.sst): second pass time = 1.29's, 83.936'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.483'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3310.803'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 279.538'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32963.998'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.936'MB/sec zip pipeline throughput = 83.936'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.89 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4443 value = 3.7005 dict = 1.00 all = 3.8058 } Zip/UnZip{ index = 0.0957 value = 0.2702 dict = 1.00 all = 0.2628 } ---------------------------- total value len = 2252.702103 GB avg = 0.200 KB (by entry num) total key len = 307.195074 GB avg = 0.027 KB total ukey len = 237.577664 GB avg = 0.015 KB total ukey num = 15.749807994 Billion total entry num = 11.291121942 Billion write speed all = 35.922271887 MB/sec (with seq num) write speed all = 34.654712374 MB/sec (without seq num) 2024/11/24-04:42:54.375677 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292359: 919300 keys, 35266546 bytes, temperature: kWarm 2024/11/24-04:42:54.375749 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394574375712, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292359, "file_size": 35266546, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4280402488, "largest_seqno": 4390540217, "table_properties": {"data_size": 29314288, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394571, "oldest_key_time": 0, "file_creation_time": 1732394571, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292359, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:54.415460 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292360.sst) /dev/shm/ToplingTemp/Topling-272325-EXLQa7 2024/11/24-04:42:55.423436 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292360.sst): first pass time = 1.01's, 133.161'MB/sec 2024/11/24-04:42:56.763428 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292360.sst): old prealloc_size = 147639500, real_size = 30262656 2024/11/24-04:42:56.763961 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292360.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932322 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0283 GB avg-val = 115.96 avg-zval = 30.31 2024/11/24-04:42:56.764778 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292360.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:42:57.591251 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292360.sst): second pass time = 1.34's, 80.737'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.528'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3359.628'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 269.445'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32296.816'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.737'MB/sec zip pipeline throughput = 80.737'MB/sec entries = 932322 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.96 avg-zval = 30.31 usrkeys = 932322 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.96 avg-zval = 30.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932322 bytes = 3263136 } seq expand size = 6526254 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.5143 value = 3.8262 dict = 1.00 all = 3.9043 } Zip/UnZip{ index = 0.0951 value = 0.2614 dict = 1.00 all = 0.2561 } ---------------------------- total value len = 2252.816742 GB avg = 0.200 KB (by entry num) total key len = 307.221179 GB avg = 0.027 KB total ukey len = 237.596310 GB avg = 0.015 KB total ukey num = 15.750740316 Billion total entry num = 11.292054264 Billion write speed all = 35.922604269 MB/sec (with seq num) write speed all = 34.654998056 MB/sec (without seq num) 2024/11/24-04:42:57.634103 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292360: 932322 keys, 34376612 bytes, temperature: kWarm 2024/11/24-04:42:57.634176 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394577634140, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292360, "file_size": 34376612, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4444082740, "table_properties": {"data_size": 28255920, "index_size": 1773440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26105016, "raw_average_key_size": 28, "raw_value_size": 108112832, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932322, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394574, "oldest_key_time": 0, "file_creation_time": 1732394574, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292360, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:42:57.669596 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292361.sst) /dev/shm/ToplingTemp/Topling-272325-K69O47 2024/11/24-04:42:58.635605 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292361.sst): first pass time = 0.97's, 138.947'MB/sec 2024/11/24-04:42:59.951161 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292361.sst): old prealloc_size = 147639500, real_size = 31327872 2024/11/24-04:42:59.951716 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292361.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.91 2024/11/24-04:42:59.952476 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292361.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:43:00.764923 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292361.sst): second pass time = 1.31's, 82.504'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.368'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3284.118'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 279.966'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32082.853'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.504'MB/sec zip pipeline throughput = 82.504'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.91 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6975 dict = 1.00 all = 3.8033 } Zip/UnZip{ index = 0.0957 value = 0.2704 dict = 1.00 all = 0.2629 } ---------------------------- total value len = 2252.931655 GB avg = 0.199 KB (by entry num) total key len = 307.246919 GB avg = 0.027 KB total ukey len = 237.614696 GB avg = 0.015 KB total ukey num = 15.751659616 Billion total entry num = 11.292973564 Billion write speed all = 35.922978150 MB/sec (with seq num) write speed all = 34.655325193 MB/sec (without seq num) 2024/11/24-04:43:00.807790 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292361: 919300 keys, 35289714 bytes, temperature: kWarm 2024/11/24-04:43:00.807860 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394580807825, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292361, "file_size": 35289714, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4268201117, "largest_seqno": 4376266558, "table_properties": {"data_size": 29337648, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394577, "oldest_key_time": 0, "file_creation_time": 1732394577, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292361, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:43:00.844684 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292362.sst) /dev/shm/ToplingTemp/Topling-272325-FgneS9 2024/11/24-04:43:01.847163 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292362.sst): first pass time = 1.00's, 133.891'MB/sec 2024/11/24-04:43:03.139718 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292362.sst): old prealloc_size = 147639500, real_size = 30246576 2024/11/24-04:43:03.140301 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292362.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929538 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0282 GB avg-val = 116.39 avg-zval = 30.36 2024/11/24-04:43:03.142322 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292362.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:43:03.956694 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292362.sst): second pass time = 1.29's, 83.752'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.906'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3202.460'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.339'MB/sec write SST data time = 0.13's, 262.459'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33115.651'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.752'MB/sec zip pipeline throughput = 83.752'MB/sec entries = 929538 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.39 avg-zval = 30.36 usrkeys = 929538 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.39 avg-zval = 30.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929538 bytes = 3137200 } seq expand size = 6506766 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0282 GB dict = 1.08 MB all = 0.0342 GB } UnZip/Zip{ index = 10.3776 value = 3.8335 dict = 1.00 all = 3.9205 } Zip/UnZip{ index = 0.0964 value = 0.2609 dict = 1.00 all = 0.2551 } ---------------------------- total value len = 2253.046352 GB avg = 0.199 KB (by entry num) total key len = 307.272946 GB avg = 0.027 KB total ukey len = 237.633287 GB avg = 0.015 KB total ukey num = 15.752589154 Billion total entry num = 11.293903102 Billion write speed all = 35.923343885 MB/sec (with seq num) write speed all = 34.655643360 MB/sec (without seq num) 2024/11/24-04:43:03.998577 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292362: 929538 keys, 34234848 bytes, temperature: kWarm 2024/11/24-04:43:03.998648 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394583998612, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292362, "file_size": 34234848, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4376266559, "largest_seqno": 4444081921, "table_properties": {"data_size": 28222560, "index_size": 1791424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26027064, "raw_average_key_size": 28, "raw_value_size": 108190696, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929538, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394580, "oldest_key_time": 0, "file_creation_time": 1732394580, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292362, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:43:04.051374 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292363.sst) /dev/shm/ToplingTemp/Topling-272325-kKziH7 2024/11/24-04:43:05.053617 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292363.sst): first pass time = 1.00's, 133.923'MB/sec 2024/11/24-04:43:06.391111 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292363.sst): old prealloc_size = 147639500, real_size = 32056768 2024/11/24-04:43:06.391676 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292363.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921038 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0301 GB avg-val = 117.72 avg-zval = 32.65 2024/11/24-04:43:06.393658 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292363.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:43:07.209903 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292363.sst): second pass time = 1.34's, 81.118'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.139'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3237.263'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 284.121'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 28735.980'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.118'MB/sec zip pipeline throughput = 81.118'MB/sec entries = 921038 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.72 avg-zval = 32.65 usrkeys = 921038 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.72 avg-zval = 32.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921038 bytes = 3223648 } seq expand size = 6447266 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5126 value = 3.6054 dict = 1.00 all = 3.7139 } Zip/UnZip{ index = 0.0951 value = 0.2774 dict = 1.00 all = 0.2693 } ---------------------------- total value len = 2253.161228 GB avg = 0.199 KB (by entry num) total key len = 307.298735 GB avg = 0.027 KB total ukey len = 237.651708 GB avg = 0.015 KB total ukey num = 15.753510192 Billion total entry num = 11.294824140 Billion write speed all = 35.923677789 MB/sec (with seq num) write speed all = 34.655931747 MB/sec (without seq num) 2024/11/24-04:43:07.254264 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292363: 921038 keys, 36139736 bytes, temperature: kWarm 2024/11/24-04:43:07.254337 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394587254300, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292363, "file_size": 36139736, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4444082559, "table_properties": {"data_size": 30074096, "index_size": 1752256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25789064, "raw_average_key_size": 28, "raw_value_size": 108428756, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921038, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394584, "oldest_key_time": 0, "file_creation_time": 1732394584, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292363, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:43:07.294667 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292364.sst) /dev/shm/ToplingTemp/Topling-272325-thzlQ9 2024/11/24-04:43:08.300549 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292364.sst): first pass time = 1.01's, 133.438'MB/sec 2024/11/24-04:43:09.611133 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292364.sst): old prealloc_size = 147639500, real_size = 30654448 2024/11/24-04:43:09.611719 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292364.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927192 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0286 GB avg-val = 116.76 avg-zval = 30.90 2024/11/24-04:43:09.612512 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292364.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:43:10.427659 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292364.sst): second pass time = 1.31's, 82.654'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.421'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3167.636'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 269.047'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 24762.236'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.654'MB/sec zip pipeline throughput = 82.654'MB/sec entries = 927192 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.76 avg-zval = 30.90 usrkeys = 927192 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.76 avg-zval = 30.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927192 bytes = 3129280 } seq expand size = 6490344 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4459 value = 3.7789 dict = 1.00 all = 3.8750 } Zip/UnZip{ index = 0.0957 value = 0.2646 dict = 1.00 all = 0.2581 } ---------------------------- total value len = 2253.275975 GB avg = 0.199 KB (by entry num) total key len = 307.324697 GB avg = 0.027 KB total ukey len = 237.670252 GB avg = 0.015 KB total ukey num = 15.754437384 Billion total entry num = 11.295751332 Billion write speed all = 35.924030135 MB/sec (with seq num) write speed all = 34.656237259 MB/sec (without seq num) 2024/11/24-04:43:10.471405 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292364: 927192 keys, 34636608 bytes, temperature: kWarm 2024/11/24-04:43:10.471474 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394590471439, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292364, "file_size": 34636608, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4366092207, "largest_seqno": 4444081940, "table_properties": {"data_size": 28647264, "index_size": 1775232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961376, "raw_average_key_size": 28, "raw_value_size": 108256476, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927192, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394587, "oldest_key_time": 0, "file_creation_time": 1732394587, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292364, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:43:10.507604 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292365.sst) /dev/shm/ToplingTemp/Topling-272325-6d9Vy6 2024/11/24-04:43:11.485050 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292365.sst): first pass time = 0.98's, 137.320'MB/sec 2024/11/24-04:43:12.802933 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292365.sst): old prealloc_size = 147639500, real_size = 32723152 2024/11/24-04:43:12.803530 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292365.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923554 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0307 GB avg-val = 117.33 avg-zval = 33.28 2024/11/24-04:43:12.805560 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292365.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:43:13.643757 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292365.sst): second pass time = 1.32's, 82.268'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.783'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3076.006'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 284.893'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 30704.754'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.268'MB/sec zip pipeline throughput = 82.268'MB/sec entries = 923554 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.33 avg-zval = 33.28 usrkeys = 923554 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.33 avg-zval = 33.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923554 bytes = 3232448 } seq expand size = 6464878 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5137 value = 3.5255 dict = 1.00 all = 3.6459 } Zip/UnZip{ index = 0.0951 value = 0.2836 dict = 1.00 all = 0.2743 } ---------------------------- total value len = 2253.390798 GB avg = 0.199 KB (by entry num) total key len = 307.350556 GB avg = 0.027 KB total ukey len = 237.688723 GB avg = 0.015 KB total ukey num = 15.755360938 Billion total entry num = 11.296674886 Billion write speed all = 35.924382924 MB/sec (with seq num) write speed all = 34.656543598 MB/sec (without seq num) 2024/11/24-04:43:13.690092 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292365: 923554 keys, 36812980 bytes, temperature: kWarm 2024/11/24-04:43:13.690162 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394593690126, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292365, "file_size": 36812980, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4444082485, "table_properties": {"data_size": 30735232, "index_size": 1756864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25859512, "raw_average_key_size": 28, "raw_value_size": 108358220, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923554, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394590, "oldest_key_time": 0, "file_creation_time": 1732394590, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292365, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:43:13.726905 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292366.sst) /dev/shm/ToplingTemp/Topling-272325-QOtHe9 2024/11/24-04:43:14.639189 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292366.sst): first pass time = 0.91's, 147.130'MB/sec 2024/11/24-04:43:15.921733 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292366.sst): old prealloc_size = 147639500, real_size = 30879344 2024/11/24-04:43:15.922352 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292366.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924686 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.15 avg-zval = 31.22 2024/11/24-04:43:15.923109 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292366.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:43:16.743738 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292366.sst): second pass time = 1.28's, 84.517'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.403'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2972.608'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 268.115'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30664.323'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.517'MB/sec zip pipeline throughput = 84.517'MB/sec entries = 924686 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.15 avg-zval = 31.22 usrkeys = 924686 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.15 avg-zval = 31.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924686 bytes = 3120832 } seq expand size = 6472802 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.3765 value = 3.7528 dict = 1.00 all = 3.8508 } Zip/UnZip{ index = 0.0964 value = 0.2665 dict = 1.00 all = 0.2597 } ---------------------------- total value len = 2253.505598 GB avg = 0.199 KB (by entry num) total key len = 307.376447 GB avg = 0.027 KB total ukey len = 237.707217 GB avg = 0.015 KB total ukey num = 15.756285624 Billion total entry num = 11.297599572 Billion write speed all = 35.924794313 MB/sec (with seq num) write speed all = 34.656906347 MB/sec (without seq num) 2024/11/24-04:43:16.788819 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292366: 924686 keys, 34854480 bytes, temperature: kWarm 2024/11/24-04:43:16.788889 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394596788854, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292366, "file_size": 34854480, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4355290422, "largest_seqno": 4444081955, "table_properties": {"data_size": 28865696, "index_size": 1782272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25891208, "raw_average_key_size": 28, "raw_value_size": 108326560, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924686, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394593, "oldest_key_time": 0, "file_creation_time": 1732394593, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292366, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:43:16.832690 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292367.sst) /dev/shm/ToplingTemp/Topling-272325-DnwCc7 2024/11/24-04:43:17.822487 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292367.sst): first pass time = 0.99's, 135.607'MB/sec 2024/11/24-04:43:19.138536 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292367.sst): old prealloc_size = 147639500, real_size = 32848672 2024/11/24-04:43:19.139100 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292367.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926040 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 116.94 avg-zval = 33.31 2024/11/24-04:43:19.140534 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292367.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:43:19.990172 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292367.sst): second pass time = 1.32's, 82.333'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.382'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3284.173'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 285.549'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33098.958'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.333'MB/sec zip pipeline throughput = 82.333'MB/sec entries = 926040 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.94 avg-zval = 33.31 usrkeys = 926040 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.94 avg-zval = 33.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926040 bytes = 3241152 } seq expand size = 6482280 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4442 value = 3.5109 dict = 1.00 all = 3.6330 } Zip/UnZip{ index = 0.0957 value = 0.2848 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 2253.620369 GB avg = 0.199 KB (by entry num) total key len = 307.402377 GB avg = 0.027 KB total ukey len = 237.725737 GB avg = 0.015 KB total ukey num = 15.757211664 Billion total entry num = 11.298525612 Billion write speed all = 35.925131997 MB/sec (with seq num) write speed all = 34.657197849 MB/sec (without seq num) 2024/11/24-04:43:20.041342 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292367: 926040 keys, 36944558 bytes, temperature: kWarm 2024/11/24-04:43:20.041412 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394600041376, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292367, "file_size": 36944558, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4444080576, "table_properties": {"data_size": 30843664, "index_size": 1773312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25929120, "raw_average_key_size": 28, "raw_value_size": 108288724, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926040, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394596, "oldest_key_time": 0, "file_creation_time": 1732394596, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292367, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:43:20.083027 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292368.sst) /dev/shm/ToplingTemp/Topling-272325-TVnOp6 2024/11/24-04:43:21.054729 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292368.sst): first pass time = 0.97's, 138.131'MB/sec 2024/11/24-04:43:22.339154 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292368.sst): old prealloc_size = 147639500, real_size = 31076704 2024/11/24-04:43:22.339700 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292368.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922156 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.55 avg-zval = 31.54 2024/11/24-04:43:22.340462 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292368.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:43:23.151027 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292368.sst): second pass time = 1.28's, 84.445'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.168'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3374.008'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.338'MB/sec write SST data time = 0.13's, 272.812'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35793.901'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.445'MB/sec zip pipeline throughput = 84.445'MB/sec entries = 922156 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.55 avg-zval = 31.54 usrkeys = 922156 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.55 avg-zval = 31.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922156 bytes = 3112288 } seq expand size = 6455092 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4456 value = 3.7275 dict = 1.00 all = 3.8298 } Zip/UnZip{ index = 0.0957 value = 0.2683 dict = 1.00 all = 0.2611 } ---------------------------- total value len = 2253.735221 GB avg = 0.199 KB (by entry num) total key len = 307.428197 GB avg = 0.027 KB total ukey len = 237.744180 GB avg = 0.015 KB total ukey num = 15.758133820 Billion total entry num = 11.299447768 Billion write speed all = 35.925512398 MB/sec (with seq num) write speed all = 34.657530986 MB/sec (without seq num) 2024/11/24-04:43:23.196992 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292368: 922156 keys, 35045758 bytes, temperature: kWarm 2024/11/24-04:43:23.197064 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394603197028, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292368, "file_size": 35045758, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4344639653, "largest_seqno": 4444081968, "table_properties": {"data_size": 29080336, "index_size": 1765632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25820368, "raw_average_key_size": 28, "raw_value_size": 108397488, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922156, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394600, "oldest_key_time": 0, "file_creation_time": 1732394600, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292368, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:43:23.233306 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292369.sst) /dev/shm/ToplingTemp/Topling-272325-cbkmL7 2024/11/24-04:43:24.237345 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292369.sst): first pass time = 1.00's, 133.683'MB/sec 2024/11/24-04:43:25.566257 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292369.sst): old prealloc_size = 147639500, real_size = 33026272 2024/11/24-04:43:25.566840 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292369.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928691 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.52 avg-zval = 33.40 2024/11/24-04:43:25.568295 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292369.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-04:43:26.434878 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292369.sst): second pass time = 1.33's, 81.478'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.746'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3187.766'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 289.112'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35604.829'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.478'MB/sec zip pipeline throughput = 81.478'MB/sec entries = 928691 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.52 avg-zval = 33.40 usrkeys = 928691 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.52 avg-zval = 33.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928691 bytes = 3250432 } seq expand size = 6500837 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4436 value = 3.4891 dict = 1.00 all = 3.6148 } Zip/UnZip{ index = 0.0958 value = 0.2866 dict = 1.00 all = 0.2766 } ---------------------------- total value len = 2253.849937 GB avg = 0.199 KB (by entry num) total key len = 307.454200 GB avg = 0.027 KB total ukey len = 237.762754 GB avg = 0.015 KB total ukey num = 15.759062511 Billion total entry num = 11.300376459 Billion write speed all = 35.925831418 MB/sec (with seq num) write speed all = 34.657804201 MB/sec (without seq num) 2024/11/24-04:43:26.483139 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292369: 928691 keys, 37129908 bytes, temperature: kWarm 2024/11/24-04:43:26.483210 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394606483174, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292369, "file_size": 37129908, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4444079967, "table_properties": {"data_size": 31015440, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003348, "raw_average_key_size": 28, "raw_value_size": 108214478, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928691, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394603, "oldest_key_time": 0, "file_creation_time": 1732394603, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292369, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:43:26.521524 139957257127488 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292370.sst) /dev/shm/ToplingTemp/Topling-272325-eeMDG6 2024/11/24-04:43:26.685972 139957257127488 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292370.sst): first pass time = 0.16's, 247.129'MB/sec 2024/11/24-04:43:27.081815 139957257127488 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292370.sst): old prealloc_size = 147639500, real_size = 11193152 2024/11/24-04:43:27.082068 139957257127488 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292370.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 278290 prefix = 4 raw-key = 0.0045 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0328 GB zip-val = 0.0106 GB avg-val = 118.00 avg-zval = 38.05 2024/11/24-04:43:27.082326 139957257127488 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292370.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0007 GB, waited 0.000 sec, Key+Value = 0.041 GB 2024/11/24-04:43:27.335867 139957257127488 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00/292370.sst): second pass time = 0.40's, 83.102'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 48.538'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2346.529'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.325'MB/sec write SST data time = 0.04's, 311.443'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.335'MB, throughput = 10480.688'MB/sec zip my value time = 0.40's, unzip length = 0.033'GB zip my value throughput = 83.102'MB/sec zip pipeline throughput = 83.102'MB/sec entries = 278290 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 38.05 usrkeys = 278290 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 38.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 278290 bytes = 869664 } seq expand size = 1948030 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0056 GB value = 0.0328 GB dict = 0.34 MB all = 0.0406 GB } __ZipSize{ index = 0.0005 GB value = 0.0106 GB dict = 0.34 MB all = 0.0123 GB } UnZip/Zip{ index = 10.4225 value = 3.1011 dict = 1.00 all = 3.2952 } Zip/UnZip{ index = 0.0959 value = 0.3225 dict = 1.00 all = 0.3035 } ---------------------------- total value len = 2253.884723 GB avg = 0.199 KB (by entry num) total key len = 307.461992 GB avg = 0.027 KB total ukey len = 237.768320 GB avg = 0.015 KB total ukey num = 15.759340801 Billion total entry num = 11.300654749 Billion write speed all = 35.925974618 MB/sec (with seq num) write speed all = 34.657932198 MB/sec (without seq num) 2024/11/24-04:43:27.356606 139957257127488 [compaction_job.cc:1948] [default] [JOB 41702] Generated table #292370: 278290 keys, 12330095 bytes, temperature: kWarm 2024/11/24-04:43:27.356664 139957257127488 EVENT_LOG_v1 {"time_micros": 1732394607356636, "cf_name": "default", "job": 41702, "event": "table_file_creation", "file_number": 292370, "file_size": 12330095, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4333609934, "largest_seqno": 4366951039, "table_properties": {"data_size": 10589360, "index_size": 534016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7792120, "raw_average_key_size": 28, "raw_value_size": 32838220, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 278290, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732394606, "oldest_key_time": 0, "file_creation_time": 1732394606, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292370, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-04:43:27.371250 139957257127488 [compaction_job.cc:703] job-41702: subcompact[0], size: 0.791892 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41702/att-00 [292348,292349,292350,292351,292352,292353,292354,292355,292356,292357,292358,292359,292360,292361,292362,292363,292364,292365,292366,292367,292368,292369,292370] 2024/11/24-04:43:27.500857 139957257127488 (Original Log Time 2024/11/24-04:42:14.865788) EVENT_LOG_v1 {"time_micros": 1732394534865746, "job": 41702, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [292323], "files_L3": [292278, 292279, 292280, 292281, 292282, 292283, 292284, 292285, 292286, 292287, 292288, 292289, 292290, 292291, 292225, 292235, 292236, 292237, 292238, 292239, 292240, 292241, 292341], "score": 0.83335, "input_data_size": 812927957, "oldest_snapshot_seqno": 4444200828} 2024/11/24-04:43:27.500863 139957257127488 (Original Log Time 2024/11/24-04:43:27.499165) [compaction_job.cc:2022] [default] [JOB 41702] Compacted 1@2 + 23@3 files to L3 => 791892287 bytes 2024/11/24-04:43:27.500865 139957257127488 (Original Log Time 2024/11/24-04:43:27.500768) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 11.2 rd, 10.9 wr, level 3, files in(1, 23) out(23 +0 blob) MB in(27.1, 748.2 +0.0 blob) out(755.2 +0.0 blob), read-write-amplify(56.5) write-amplify(27.9) OK, records in: 20780048, records dropped: 100410 output_compression: Snappy 2024/11/24-04:43:27.500869 139957257127488 (Original Log Time 2024/11/24-04:43:27.500825) EVENT_LOG_v1 {"time_micros": 1732394607500796, "cf": "default", "job": 41702, "event": "compaction_finished", "compaction_time_micros": 72505470, "compaction_time_cpu_micros": 31105051, "output_level": 3, "num_output_files": 23, "total_output_size": 791892287, "num_input_records": 20780048, "num_output_records": 20679638, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 550207227, "file_range_sync_nanos": 0, "file_fsync_nanos": 15602, "file_prepare_write_nanos": 60829, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/24-04:43:27.630613 139957257127488 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732394534,"executesMs":72754,"compactionJobId":41702,"attempt":0,"compactionInputRawBytes":3007680216,"compactionInputZipBytes":812927957,"compactionOutputRawBytes":0,"compactionOutputZipBytes":791892287,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-8bd8578d6909b78d38d9b35be103392f-cfd2fda72744be85-00"}}, response = 2024/11/24-04:43:27.630705 139957257127488 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41702/att-00: olev 3, work 72.755 s, result 7.701 ms, install 1.720 ms, input{raw 2.801 GiB zip 775.269 MiB}