2024/11/23-20:14:07.799038 139957676762688 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00: fork to child time = 0.034721 sec 2024/11/23-20:14:07.799995 139957676762688 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00 2024/11/23-20:14:07.808092 139957676762688 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00 2024/11/23-20:14:07.809318 139957676762688 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-20:14:07.809686 139957676762688 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-20:14:07.859201 139957676762688 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34971/att-00/MANIFEST-206347 2024/11/23-20:14:07.885167 139957676762688 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34971/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 278858, last_sequence is 4238082436, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 278714 2024/11/23-20:14:07.885176 139957676762688 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-20:14:07.885474 139957676762688 [version_set.cc:5721] Creating manifest 278858 2024/11/23-20:14:07.917066 139957676762688 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-20:14:07.939772 139957676762688 [compaction_job.cc:2437] [default] [JOB 34971] Compacting 1@2 + 20@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-20:14:07.939798 139957676762688 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 26177 Base level 2, inputs: [278848(27MB)], [278807(36MB) 278808(36MB) 278809(35MB) 278810(36MB) 278811(38MB) 278812(4534KB) 278784(38MB) 278785(38MB) 278786(36MB) 278787(35MB) 278788(34MB) 278789(36MB) 278790(34MB) 278791(37MB) 278792(32MB) 278793(33MB) 278794(36MB) 278795(37MB) 278796(36MB) 278797(37MB)] 2024/11/23-20:14:07.948577 139957676762688 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:14:07.948620 139957676762688 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:14:07.948649 139957676762688 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:14:07.948658 139957676762688 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-20:14:07.948674 139957676762688 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-20:14:08.590239 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278859.sst) /dev/shm/ToplingTemp/Topling-176930-EWQzoP 2024/11/23-20:14:09.467086 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278859.sst): first pass time = 0.88's, 153.078'MB/sec 2024/11/23-20:14:11.005615 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278859.sst): old prealloc_size = 147639500, real_size = 34512352 2024/11/23-20:14:11.006294 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278859.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927200 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 116.76 avg-zval = 35.07 2024/11/23-20:14:11.007166 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278859.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:11.876821 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278859.sst): second pass time = 1.54's, 70.422'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 44.188'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2687.299'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.14's, 276.696'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 38168.268'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.422'MB/sec zip pipeline throughput = 70.422'MB/sec entries = 927200 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 35.07 usrkeys = 927200 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 35.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927200 bytes = 3129312 } seq expand size = 6490400 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5142 value = 3.3293 dict = 1.00 all = 3.4865 } Zip/UnZip{ index = 0.0951 value = 0.3004 dict = 1.00 all = 0.2868 } ---------------------------- total value len = 8.963366 GB avg = 0.168 KB (by entry num) total key len = 1.466783 GB avg = 0.028 KB total ukey len = 1.016422 GB avg = 0.015 KB total ukey num = 0.066477635 Billion total entry num = 0.053285364 Billion write speed all = 0.256019529 MB/sec (with seq num) write speed all = 0.245555944 MB/sec (without seq num) 2024/11/23-20:14:11.924076 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278859: 927200 keys, 38496316 bytes, temperature: kWarm 2024/11/23-20:14:11.924173 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364051924117, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278859, "file_size": 38496316, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4136224572, "largest_seqno": 4238004547, "table_properties": {"data_size": 32516688, "index_size": 1763712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25961600, "raw_average_key_size": 28, "raw_value_size": 108256232, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927200, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364048, "oldest_key_time": 0, "file_creation_time": 1732364048, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278859, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:11.961436 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278860.sst) /dev/shm/ToplingTemp/Topling-176930-8HiooO 2024/11/23-20:14:13.519103 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278860.sst): first pass time = 1.56's, 86.169'MB/sec 2024/11/23-20:14:15.134738 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278860.sst): old prealloc_size = 147639500, real_size = 34091600 2024/11/23-20:14:15.135312 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278860.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927337 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0321 GB avg-val = 116.73 avg-zval = 34.61 2024/11/23-20:14:15.136120 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278860.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:16.440471 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278860.sst): second pass time = 1.61's, 67.033'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.31's, 29.161'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3199.489'MB/sec (index lex order gen) rebuild zvType time = 1.06's, 0.220'MB/sec write SST data time = 0.25's, 152.603'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25532.117'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.033'MB/sec zip pipeline throughput = 67.033'MB/sec entries = 927337 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 34.61 usrkeys = 927337 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 34.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927337 bytes = 3129776 } seq expand size = 6491359 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5131 value = 3.3728 dict = 1.00 all = 3.5250 } Zip/UnZip{ index = 0.0951 value = 0.2965 dict = 1.00 all = 0.2837 } ---------------------------- total value len = 9.078110 GB avg = 0.167 KB (by entry num) total key len = 1.492748 GB avg = 0.028 KB total ukey len = 1.034969 GB avg = 0.015 KB total ukey num = 0.067404972 Billion total entry num = 0.054212701 Billion write speed all = 0.259444326 MB/sec (with seq num) write speed all = 0.248799834 MB/sec (without seq num) 2024/11/23-20:14:16.494357 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278860: 927337 keys, 38075710 bytes, temperature: kWarm 2024/11/23-20:14:16.494448 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364056494401, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278860, "file_size": 38075710, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4168606255, "largest_seqno": 4238004938, "table_properties": {"data_size": 32095424, "index_size": 1764160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965436, "raw_average_key_size": 28, "raw_value_size": 108252310, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927337, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364051, "oldest_key_time": 0, "file_creation_time": 1732364051, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278860, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:16.554477 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278861.sst) /dev/shm/ToplingTemp/Topling-176930-tHNKUQ 2024/11/23-20:14:18.061029 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278861.sst): first pass time = 1.51's, 89.093'MB/sec 2024/11/23-20:14:19.466126 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278861.sst): old prealloc_size = 147639500, real_size = 33732096 2024/11/23-20:14:19.466801 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923124 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.40 avg-zval = 34.38 2024/11/23-20:14:19.467889 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278861.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:20.552845 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278861.sst): second pass time = 1.40's, 77.177'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.09's, 34.697'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2711.535'MB/sec (index lex order gen) rebuild zvType time = 0.93's, 0.249'MB/sec write SST data time = 0.16's, 237.623'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35836.979'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.177'MB/sec zip pipeline throughput = 77.177'MB/sec entries = 923124 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.40 avg-zval = 34.38 usrkeys = 923124 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.40 avg-zval = 34.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923124 bytes = 3115552 } seq expand size = 6461868 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4433 value = 3.4150 dict = 1.00 all = 3.5596 } Zip/UnZip{ index = 0.0958 value = 0.2928 dict = 1.00 all = 0.2809 } ---------------------------- total value len = 9.192942 GB avg = 0.167 KB (by entry num) total key len = 1.518596 GB avg = 0.028 KB total ukey len = 1.053431 GB avg = 0.015 KB total ukey num = 0.068328096 Billion total entry num = 0.055135825 Billion write speed all = 0.262870544 MB/sec (with seq num) write speed all = 0.252045891 MB/sec (without seq num) 2024/11/23-20:14:20.602338 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278861: 923124 keys, 37705558 bytes, temperature: kWarm 2024/11/23-20:14:20.602407 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364060602371, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278861, "file_size": 37705558, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4155809461, "largest_seqno": 4238004753, "table_properties": {"data_size": 31733232, "index_size": 1767872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847472, "raw_average_key_size": 28, "raw_value_size": 108370268, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923124, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364056, "oldest_key_time": 0, "file_creation_time": 1732364056, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:20.639211 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278862.sst) /dev/shm/ToplingTemp/Topling-176930-eo4h8N 2024/11/23-20:14:21.961957 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278862.sst): first pass time = 1.32's, 101.473'MB/sec 2024/11/23-20:14:23.464600 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278862.sst): old prealloc_size = 147639500, real_size = 33925392 2024/11/23-20:14:23.465331 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930546 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0319 GB avg-val = 116.24 avg-zval = 34.31 2024/11/23-20:14:23.466147 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278862.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:24.333753 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278862.sst): second pass time = 1.50's, 72.056'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.493'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2495.798'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.14's, 273.382'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34609.642'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.056'MB/sec zip pipeline throughput = 72.056'MB/sec entries = 930546 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.24 avg-zval = 34.31 usrkeys = 930546 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.24 avg-zval = 34.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930546 bytes = 3024288 } seq expand size = 6513822 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5140 value = 3.3883 dict = 1.00 all = 3.5506 } Zip/UnZip{ index = 0.0951 value = 0.2951 dict = 1.00 all = 0.2816 } ---------------------------- total value len = 9.307619 GB avg = 0.166 KB (by entry num) total key len = 1.544651 GB avg = 0.028 KB total ukey len = 1.072042 GB avg = 0.015 KB total ukey num = 0.069258642 Billion total entry num = 0.056066371 Billion write speed all = 0.266299512 MB/sec (with seq num) write speed all = 0.255293190 MB/sec (without seq num) 2024/11/23-20:14:24.384270 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278862: 930546 keys, 37801130 bytes, temperature: kWarm 2024/11/23-20:14:24.384346 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364064384301, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278862, "file_size": 37801130, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4191256266, "largest_seqno": 4238005023, "table_properties": {"data_size": 31922432, "index_size": 1770112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26055288, "raw_average_key_size": 28, "raw_value_size": 108162516, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930546, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364060, "oldest_key_time": 0, "file_creation_time": 1732364060, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:24.420930 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278863.sst) /dev/shm/ToplingTemp/Topling-176930-6PvfCP 2024/11/23-20:14:25.569493 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278863.sst): first pass time = 1.15's, 116.860'MB/sec 2024/11/23-20:14:27.449507 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278863.sst): old prealloc_size = 147639500, real_size = 35694400 2024/11/23-20:14:27.450109 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278863.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931070 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1081 GB zip-val = 0.0337 GB avg-val = 116.15 avg-zval = 36.20 2024/11/23-20:14:27.452085 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278863.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:28.465718 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278863.sst): second pass time = 1.88's, 57.549'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 39.404'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3048.369'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.24's, 169.857'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33455.200'MB/sec zip my value time = 1.88's, unzip length = 0.108'GB zip my value throughput = 57.549'MB/sec zip pipeline throughput = 127.594'MB/sec entries = 931070 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.15 avg-zval = 36.20 usrkeys = 931070 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.15 avg-zval = 36.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931070 bytes = 3491520 } seq expand size = 6517490 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.5823 value = 3.2090 dict = 1.00 all = 3.3520 } Zip/UnZip{ index = 0.0945 value = 0.3116 dict = 1.00 all = 0.2983 } ---------------------------- total value len = 9.422284 GB avg = 0.165 KB (by entry num) total key len = 1.570721 GB avg = 0.028 KB total ukey len = 1.090664 GB avg = 0.016 KB total ukey num = 0.070189712 Billion total entry num = 0.056997441 Billion write speed all = 0.269725612 MB/sec (with seq num) write speed all = 0.258537646 MB/sec (without seq num) 2024/11/23-20:14:28.517670 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278863: 931070 keys, 40041096 bytes, temperature: kWarm 2024/11/23-20:14:28.517777 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364068517716, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278863, "file_size": 40041096, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4238004815, "table_properties": {"data_size": 33701744, "index_size": 1759680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26069960, "raw_average_key_size": 28, "raw_value_size": 108147856, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931070, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364064, "oldest_key_time": 0, "file_creation_time": 1732364064, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278863, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:28.586798 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278864.sst) /dev/shm/ToplingTemp/Topling-176930-hw5EfN 2024/11/23-20:14:29.575674 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278864.sst): first pass time = 0.99's, 135.734'MB/sec 2024/11/23-20:14:31.139025 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278864.sst): old prealloc_size = 147639500, real_size = 36276048 2024/11/23-20:14:31.139560 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278864.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934956 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0342 GB avg-val = 115.56 avg-zval = 36.62 2024/11/23-20:14:31.140187 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278864.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:32.089333 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278864.sst): second pass time = 1.56's, 69.141'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 41.706'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3503.881'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.286'MB/sec write SST data time = 0.13's, 298.115'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33252.028'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.141'MB/sec zip pipeline throughput = 69.141'MB/sec entries = 934956 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.56 avg-zval = 36.62 usrkeys = 934956 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.26 avg-zval = 36.62 TagRS{ kinds = RS_Key0_TagN bytes = 146200 } TagArray{ size = 628164 bytes = 2355632 } seq expand size = 4397148 multi value expand size = 0 cnt WriteAppend = 21 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0342 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.3773 value = 3.1553 dict = 1.00 all = 3.3866 } Zip/UnZip{ index = 0.0964 value = 0.3169 dict = 1.00 all = 0.2953 } ---------------------------- total value len = 9.534720 GB avg = 0.165 KB (by entry num) total key len = 1.596900 GB avg = 0.028 KB total ukey len = 1.109363 GB avg = 0.016 KB total ukey num = 0.071124668 Billion total entry num = 0.057932397 Billion write speed all = 0.273102404 MB/sec (with seq num) write speed all = 0.261731928 MB/sec (without seq num) 2024/11/23-20:14:32.171585 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278864: 934956 keys, 39632202 bytes, temperature: kWarm 2024/11/23-20:14:32.171661 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364072171622, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278864, "file_size": 39632202, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34240192, "index_size": 1801920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26178768, "raw_average_key_size": 28, "raw_value_size": 108039084, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934956, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364068, "oldest_key_time": 0, "file_creation_time": 1732364068, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278864, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:32.210389 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278865.sst) /dev/shm/ToplingTemp/Topling-176930-vmHAIN 2024/11/23-20:14:33.150993 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278865.sst): first pass time = 0.94's, 142.699'MB/sec 2024/11/23-20:14:34.706791 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278865.sst): old prealloc_size = 147639500, real_size = 36017312 2024/11/23-20:14:34.707357 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278865.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926383 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0340 GB avg-val = 116.88 avg-zval = 36.71 2024/11/23-20:14:34.709250 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278865.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:35.715323 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278865.sst): second pass time = 1.55's, 69.639'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 39.890'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3276.632'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.264'MB/sec write SST data time = 0.13's, 306.951'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 38272.811'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.639'MB/sec zip pipeline throughput = 69.639'MB/sec entries = 926383 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.88 avg-zval = 36.71 usrkeys = 926383 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.88 avg-zval = 36.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926383 bytes = 3358160 } seq expand size = 6484681 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.4435 value = 3.1836 dict = 1.00 all = 3.3362 } Zip/UnZip{ index = 0.0958 value = 0.3141 dict = 1.00 all = 0.2997 } ---------------------------- total value len = 9.649484 GB avg = 0.164 KB (by entry num) total key len = 1.622838 GB avg = 0.028 KB total ukey len = 1.127890 GB avg = 0.016 KB total ukey num = 0.072051051 Billion total entry num = 0.058858780 Billion write speed all = 0.276529790 MB/sec (with seq num) write speed all = 0.264978519 MB/sec (without seq num) 2024/11/23-20:14:35.773100 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278865: 926383 keys, 40230230 bytes, temperature: kWarm 2024/11/23-20:14:35.773172 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364075773137, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278865, "file_size": 40230230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 34011472, "index_size": 1774080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25938724, "raw_average_key_size": 28, "raw_value_size": 108279110, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926383, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364072, "oldest_key_time": 0, "file_creation_time": 1732364072, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278865, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:35.810323 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278866.sst) /dev/shm/ToplingTemp/Topling-176930-1sWuAQ 2024/11/23-20:14:37.020477 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278866.sst): first pass time = 1.21's, 110.914'MB/sec 2024/11/23-20:14:38.457786 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278866.sst): old prealloc_size = 147639500, real_size = 32868656 2024/11/23-20:14:38.458489 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278866.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928644 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.53 avg-zval = 33.24 2024/11/23-20:14:38.460598 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278866.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:39.433112 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278866.sst): second pass time = 1.44's, 75.340'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 38.147'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2597.353'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.16's, 239.256'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34981.957'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.340'MB/sec zip pipeline throughput = 75.340'MB/sec entries = 928644 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.53 avg-zval = 33.24 usrkeys = 928644 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.53 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928644 bytes = 3482432 } seq expand size = 6500508 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5145 value = 3.5055 dict = 1.00 all = 3.6075 } Zip/UnZip{ index = 0.0951 value = 0.2853 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 9.764200 GB avg = 0.163 KB (by entry num) total key len = 1.648840 GB avg = 0.028 KB total ukey len = 1.146463 GB avg = 0.016 KB total ukey num = 0.072979695 Billion total entry num = 0.059787424 Billion write speed all = 0.279956323 MB/sec (with seq num) write speed all = 0.268223872 MB/sec (without seq num) 2024/11/23-20:14:39.482987 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278866: 928644 keys, 37204744 bytes, temperature: kWarm 2024/11/23-20:14:39.483064 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364079483027, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278866, "file_size": 37204744, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30869920, "index_size": 1766400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002032, "raw_average_key_size": 28, "raw_value_size": 108215788, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928644, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364075, "oldest_key_time": 0, "file_creation_time": 1732364075, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278866, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:39.521022 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278867.sst) /dev/shm/ToplingTemp/Topling-176930-t19YKP 2024/11/23-20:14:40.778515 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278867.sst): first pass time = 1.26's, 106.738'MB/sec 2024/11/23-20:14:42.226349 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278867.sst): old prealloc_size = 147639500, real_size = 32811456 2024/11/23-20:14:42.227045 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278867.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930517 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.24 avg-zval = 33.08 2024/11/23-20:14:42.227835 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278867.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:43.181049 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278867.sst): second pass time = 1.45's, 74.756'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.355'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2653.162'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.16's, 235.925'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32725.601'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.756'MB/sec zip pipeline throughput = 74.756'MB/sec entries = 930517 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.24 avg-zval = 33.08 usrkeys = 930517 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.86 avg-zval = 33.08 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 747520 bytes = 2803216 } seq expand size = 5232640 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.3778 value = 3.5135 dict = 1.00 all = 3.6656 } Zip/UnZip{ index = 0.0964 value = 0.2846 dict = 1.00 all = 0.2728 } ---------------------------- total value len = 9.877596 GB avg = 0.163 KB (by entry num) total key len = 1.674895 GB avg = 0.028 KB total ukey len = 1.165074 GB avg = 0.016 KB total ukey num = 0.073910212 Billion total entry num = 0.060717941 Billion write speed all = 0.283350924 MB/sec (with seq num) write speed all = 0.271436968 MB/sec (without seq num) 2024/11/23-20:14:43.228773 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278867: 930517 keys, 36615670 bytes, temperature: kWarm 2024/11/23-20:14:43.228840 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364083228805, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278867, "file_size": 36615670, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 30785392, "index_size": 1793280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26054476, "raw_average_key_size": 28, "raw_value_size": 108163302, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930517, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364079, "oldest_key_time": 0, "file_creation_time": 1732364079, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278867, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:43.269648 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278868.sst) /dev/shm/ToplingTemp/Topling-176930-QYspGM 2024/11/23-20:14:44.202053 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278868.sst): first pass time = 0.93's, 143.954'MB/sec 2024/11/23-20:14:45.516390 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278868.sst): old prealloc_size = 147639500, real_size = 30918816 2024/11/23-20:14:45.516931 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278868.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932181 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0289 GB avg-val = 115.98 avg-zval = 31.02 2024/11/23-20:14:45.517768 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278868.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:46.630472 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278868.sst): second pass time = 1.31's, 82.334'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 31.551'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3420.615'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.266'MB/sec write SST data time = 0.24's, 148.639'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29625.986'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.334'MB/sec zip pipeline throughput = 82.334'MB/sec entries = 932181 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.98 avg-zval = 31.02 usrkeys = 932181 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.98 avg-zval = 31.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932181 bytes = 3379168 } seq expand size = 6525267 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5134 value = 3.7395 dict = 1.00 all = 3.8186 } Zip/UnZip{ index = 0.0951 value = 0.2674 dict = 1.00 all = 0.2619 } ---------------------------- total value len = 9.992238 GB avg = 0.162 KB (by entry num) total key len = 1.700996 GB avg = 0.028 KB total ukey len = 1.183717 GB avg = 0.016 KB total ukey num = 0.074842393 Billion total entry num = 0.061650122 Billion write speed all = 0.286778705 MB/sec (with seq num) write speed all = 0.274682861 MB/sec (without seq num) 2024/11/23-20:14:46.680804 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278868: 932181 keys, 35148796 bytes, temperature: kWarm 2024/11/23-20:14:46.680888 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364086680839, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278868, "file_size": 35148796, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 28912272, "index_size": 1773312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26101068, "raw_average_key_size": 28, "raw_value_size": 108116802, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932181, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364083, "oldest_key_time": 0, "file_creation_time": 1732364083, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278868, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:46.738768 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278869.sst) /dev/shm/ToplingTemp/Topling-176930-c2r7tN 2024/11/23-20:14:47.729764 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278869.sst): first pass time = 0.99's, 135.445'MB/sec 2024/11/23-20:14:49.268289 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278869.sst): old prealloc_size = 147639500, real_size = 32725856 2024/11/23-20:14:49.268837 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278869.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 115.96 avg-zval = 32.92 2024/11/23-20:14:49.269646 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278869.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:50.249400 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278869.sst): second pass time = 1.54's, 70.306'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 37.670'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3388.923'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.23's, 159.190'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25753.487'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.306'MB/sec zip pipeline throughput = 70.306'MB/sec entries = 932309 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.96 avg-zval = 32.92 usrkeys = 932309 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.96 avg-zval = 32.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932309 bytes = 3379632 } seq expand size = 6526163 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.3774 value = 3.5221 dict = 1.00 all = 3.6318 } Zip/UnZip{ index = 0.0964 value = 0.2839 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 10.106878 GB avg = 0.161 KB (by entry num) total key len = 1.727101 GB avg = 0.028 KB total ukey len = 1.202363 GB avg = 0.016 KB total ukey num = 0.075774702 Billion total entry num = 0.062582431 Billion write speed all = 0.290204720 MB/sec (with seq num) write speed all = 0.277927045 MB/sec (without seq num) 2024/11/23-20:14:50.298535 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278869: 932309 keys, 36956368 bytes, temperature: kWarm 2024/11/23-20:14:50.298601 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364090298569, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278869, "file_size": 36956368, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3521157726, "largest_seqno": 4012776268, "table_properties": {"data_size": 30695824, "index_size": 1796800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26104652, "raw_average_key_size": 28, "raw_value_size": 108113186, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364086, "oldest_key_time": 0, "file_creation_time": 1732364086, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278869, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:50.334419 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278870.sst) /dev/shm/ToplingTemp/Topling-176930-VGh2zN 2024/11/23-20:14:51.335267 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278870.sst): first pass time = 1.00's, 134.108'MB/sec 2024/11/23-20:14:52.660524 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278870.sst): old prealloc_size = 147639500, real_size = 33674144 2024/11/23-20:14:52.661070 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278870.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922525 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.49 avg-zval = 34.34 2024/11/23-20:14:52.661895 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278870.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:53.502691 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278870.sst): second pass time = 1.32's, 81.836'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.114'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3393.687'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 297.646'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35670.223'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.836'MB/sec zip pipeline throughput = 81.836'MB/sec entries = 922525 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.49 avg-zval = 34.34 usrkeys = 922525 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.49 avg-zval = 34.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922525 bytes = 3459488 } seq expand size = 6457675 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.4430 value = 3.4217 dict = 1.00 all = 3.5328 } Zip/UnZip{ index = 0.0958 value = 0.2923 dict = 1.00 all = 0.2831 } ---------------------------- total value len = 10.221722 GB avg = 0.161 KB (by entry num) total key len = 1.752931 GB avg = 0.028 KB total ukey len = 1.220814 GB avg = 0.016 KB total ukey num = 0.076697227 Billion total entry num = 0.063504956 Billion write speed all = 0.293631079 MB/sec (with seq num) write speed all = 0.281173413 MB/sec (without seq num) 2024/11/23-20:14:53.550540 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278870: 922525 keys, 37992192 bytes, temperature: kWarm 2024/11/23-20:14:53.550612 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364093550577, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278870, "file_size": 37992192, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 31676560, "index_size": 1766784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25830700, "raw_average_key_size": 28, "raw_value_size": 108387078, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922525, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364090, "oldest_key_time": 0, "file_creation_time": 1732364090, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278870, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:53.588525 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278871.sst) /dev/shm/ToplingTemp/Topling-176930-GTxsPO 2024/11/23-20:14:54.514534 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278871.sst): first pass time = 0.93's, 144.948'MB/sec 2024/11/23-20:14:55.888123 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278871.sst): old prealloc_size = 147639500, real_size = 33814032 2024/11/23-20:14:55.888680 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278871.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932400 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0318 GB avg-val = 115.95 avg-zval = 34.11 2024/11/23-20:14:55.889502 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278871.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:56.742218 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278871.sst): second pass time = 1.37's, 78.777'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.545'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3327.459'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 290.548'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29278.190'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.777'MB/sec zip pipeline throughput = 78.777'MB/sec entries = 932400 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.95 avg-zval = 34.11 usrkeys = 932400 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.95 avg-zval = 34.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932400 bytes = 3379968 } seq expand size = 6526800 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5140 value = 3.3989 dict = 1.00 all = 3.5279 } Zip/UnZip{ index = 0.0951 value = 0.2942 dict = 1.00 all = 0.2835 } ---------------------------- total value len = 10.336360 GB avg = 0.160 KB (by entry num) total key len = 1.779038 GB avg = 0.028 KB total ukey len = 1.239462 GB avg = 0.016 KB total ukey num = 0.077629627 Billion total entry num = 0.064437356 Billion write speed all = 0.297058688 MB/sec (with seq num) write speed all = 0.284419119 MB/sec (without seq num) 2024/11/23-20:14:56.791062 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278871: 932400 keys, 38044554 bytes, temperature: kWarm 2024/11/23-20:14:56.791131 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364096791096, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278871, "file_size": 38044554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 31807104, "index_size": 1773632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26107200, "raw_average_key_size": 28, "raw_value_size": 108110644, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932400, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364093, "oldest_key_time": 0, "file_creation_time": 1732364093, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278871, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:56.831544 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278872.sst) /dev/shm/ToplingTemp/Topling-176930-eVHRnM 2024/11/23-20:14:57.821153 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278872.sst): first pass time = 0.99's, 135.633'MB/sec 2024/11/23-20:14:59.140846 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278872.sst): old prealloc_size = 147639500, real_size = 32569456 2024/11/23-20:14:59.141447 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278872.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926654 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 116.84 avg-zval = 32.99 2024/11/23-20:14:59.142281 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278872.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:59.981635 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278872.sst): second pass time = 1.32's, 82.091'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.888'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3049.628'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 284.727'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32180.898'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.091'MB/sec zip pipeline throughput = 82.091'MB/sec entries = 926654 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.84 avg-zval = 32.99 usrkeys = 926654 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.84 avg-zval = 32.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926654 bytes = 3474960 } seq expand size = 6486578 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5126 value = 3.5412 dict = 1.00 all = 3.6374 } Zip/UnZip{ index = 0.0951 value = 0.2824 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 10.451118 GB avg = 0.160 KB (by entry num) total key len = 1.804985 GB avg = 0.028 KB total ukey len = 1.257995 GB avg = 0.016 KB total ukey num = 0.078556281 Billion total entry num = 0.065364010 Billion write speed all = 0.300484767 MB/sec (with seq num) write speed all = 0.287664450 MB/sec (without seq num) 2024/11/23-20:15:00.028448 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278872: 926654 keys, 36899178 bytes, temperature: kWarm 2024/11/23-20:15:00.028519 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364100028482, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278872, "file_size": 36899178, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 30574688, "index_size": 1762944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946312, "raw_average_key_size": 28, "raw_value_size": 108271504, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926654, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364096, "oldest_key_time": 0, "file_creation_time": 1732364096, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278872, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:15:00.077684 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278873.sst) /dev/shm/ToplingTemp/Topling-176930-pNVURP 2024/11/23-20:15:01.015405 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278873.sst): first pass time = 0.94's, 143.138'MB/sec 2024/11/23-20:15:02.338119 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278873.sst): old prealloc_size = 147639500, real_size = 33030320 2024/11/23-20:15:02.338691 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278873.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924575 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.17 avg-zval = 33.55 2024/11/23-20:15:02.339537 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278873.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:15:03.183504 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278873.sst): second pass time = 1.32's, 81.945'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.186'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3240.852'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 292.290'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31890.147'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.945'MB/sec zip pipeline throughput = 81.945'MB/sec entries = 924575 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.17 avg-zval = 33.55 usrkeys = 924575 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.17 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924575 bytes = 3467168 } seq expand size = 6472025 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.3749 value = 3.4926 dict = 1.00 all = 3.5932 } Zip/UnZip{ index = 0.0964 value = 0.2863 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 10.565920 GB avg = 0.159 KB (by entry num) total key len = 1.830873 GB avg = 0.028 KB total ukey len = 1.276486 GB avg = 0.016 KB total ukey num = 0.079480856 Billion total entry num = 0.066288585 Billion write speed all = 0.303910225 MB/sec (with seq num) write speed all = 0.290909586 MB/sec (without seq num) 2024/11/23-20:15:03.229857 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278873: 924575 keys, 37353284 bytes, temperature: kWarm 2024/11/23-20:15:03.229927 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364103229891, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278873, "file_size": 37353284, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 31016672, "index_size": 1782336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25888100, "raw_average_key_size": 28, "raw_value_size": 108329762, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924575, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364100, "oldest_key_time": 0, "file_creation_time": 1732364100, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278873, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:15:03.268526 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278874.sst) /dev/shm/ToplingTemp/Topling-176930-B5aRiN 2024/11/23-20:15:04.226249 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278874.sst): first pass time = 0.96's, 140.148'MB/sec 2024/11/23-20:15:05.691571 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278874.sst): old prealloc_size = 147639500, real_size = 35982928 2024/11/23-20:15:05.692134 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278874.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930633 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0340 GB avg-val = 116.22 avg-zval = 36.51 2024/11/23-20:15:05.692461 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278874.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:15:06.428512 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278874.sst): second pass time = 1.46's, 73.851'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 51.332'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3293.817'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.382'MB/sec write SST data time = 0.13's, 297.685'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25128.173'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 73.851'MB/sec zip pipeline throughput = 73.851'MB/sec entries = 930633 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 36.51 usrkeys = 930633 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.17 avg-zval = 36.51 TagRS{ kinds = RS_Key0_TagN bytes = 145520 } TagArray{ size = 259413 bytes = 843104 } seq expand size = 1815891 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5119 value = 3.1831 dict = 1.00 all = 3.5482 } Zip/UnZip{ index = 0.0951 value = 0.3142 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 10.675896 GB avg = 0.159 KB (by entry num) total key len = 1.856931 GB avg = 0.028 KB total ukey len = 1.295099 GB avg = 0.016 KB total ukey num = 0.080411489 Billion total entry num = 0.067219218 Billion write speed all = 0.307220684 MB/sec (with seq num) write speed all = 0.294038576 MB/sec (without seq num) 2024/11/23-20:15:06.477215 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278874: 930633 keys, 37827465 bytes, temperature: kWarm 2024/11/23-20:15:06.477293 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364106477257, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278874, "file_size": 37827465, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 33979456, "index_size": 1770624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26057724, "raw_average_key_size": 28, "raw_value_size": 108160138, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930633, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364103, "oldest_key_time": 0, "file_creation_time": 1732364103, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278874, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:15:06.515889 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278875.sst) /dev/shm/ToplingTemp/Topling-176930-onV2aQ 2024/11/23-20:15:07.429001 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278875.sst): first pass time = 0.91's, 146.996'MB/sec 2024/11/23-20:15:08.891606 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278875.sst): old prealloc_size = 147639500, real_size = 37273872 2024/11/23-20:15:08.892183 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278875.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925479 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0353 GB avg-val = 117.03 avg-zval = 38.10 2024/11/23-20:15:08.892585 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278875.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:15:09.623293 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278875.sst): second pass time = 1.46's, 74.120'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 53.828'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3227.327'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.382'MB/sec write SST data time = 0.13's, 313.274'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35228.993'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.120'MB/sec zip pipeline throughput = 74.120'MB/sec entries = 925479 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.03 avg-zval = 38.10 usrkeys = 925479 avg-key = 20.00 avg-zkey = 1.93 avg-val = 119.26 avg-zval = 38.10 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 295106 bytes = 1106656 } seq expand size = 2065742 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0353 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.3787 value = 3.0717 dict = 1.00 all = 3.4079 } Zip/UnZip{ index = 0.0964 value = 0.3256 dict = 1.00 all = 0.2934 } ---------------------------- total value len = 10.786266 GB avg = 0.158 KB (by entry num) total key len = 1.882844 GB avg = 0.028 KB total ukey len = 1.313609 GB avg = 0.016 KB total ukey num = 0.081336968 Billion total entry num = 0.068144697 Billion write speed all = 0.310537119 MB/sec (with seq num) write speed all = 0.297174565 MB/sec (without seq num) 2024/11/23-20:15:09.672591 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278875: 925479 keys, 39383813 bytes, temperature: kWarm 2024/11/23-20:15:09.672662 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364109672627, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278875, "file_size": 39383813, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4156027118, "table_properties": {"data_size": 35258880, "index_size": 1783424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25913412, "raw_average_key_size": 28, "raw_value_size": 108304326, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925479, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364106, "oldest_key_time": 0, "file_creation_time": 1732364106, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278875, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:15:09.710353 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278876.sst) /dev/shm/ToplingTemp/Topling-176930-9qNQbO 2024/11/23-20:15:10.691527 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278876.sst): first pass time = 0.98's, 136.798'MB/sec 2024/11/23-20:15:11.995684 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278876.sst): old prealloc_size = 147639500, real_size = 33293520 2024/11/23-20:15:11.996278 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278876.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927256 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.75 avg-zval = 33.75 2024/11/23-20:15:11.997041 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278876.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:15:12.851362 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278876.sst): second pass time = 1.30's, 83.056'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.566'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3116.069'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 294.881'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30586.390'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.056'MB/sec zip pipeline throughput = 83.056'MB/sec entries = 927256 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 33.75 usrkeys = 927256 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 33.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927256 bytes = 3129504 } seq expand size = 6490792 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5137 value = 3.4589 dict = 1.00 all = 3.6005 } Zip/UnZip{ index = 0.0951 value = 0.2891 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 10.901011 GB avg = 0.158 KB (by entry num) total key len = 1.908807 GB avg = 0.028 KB total ukey len = 1.332154 GB avg = 0.016 KB total ukey num = 0.082264224 Billion total entry num = 0.069071953 Billion write speed all = 0.313961237 MB/sec (with seq num) write speed all = 0.300417928 MB/sec (without seq num) 2024/11/23-20:15:12.897818 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278876: 927256 keys, 37277564 bytes, temperature: kWarm 2024/11/23-20:15:12.897891 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364112897854, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278876, "file_size": 37277564, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4156027119, "largest_seqno": 4238003028, "table_properties": {"data_size": 31297600, "index_size": 1763904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963168, "raw_average_key_size": 28, "raw_value_size": 108254640, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927256, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364109, "oldest_key_time": 0, "file_creation_time": 1732364109, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278876, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:15:12.936112 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278877.sst) /dev/shm/ToplingTemp/Topling-176930-a0hLJM 2024/11/23-20:15:13.873596 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278877.sst): first pass time = 0.94's, 143.174'MB/sec 2024/11/23-20:15:15.196631 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278877.sst): old prealloc_size = 147639500, real_size = 34438976 2024/11/23-20:15:15.197190 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278877.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928686 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.52 avg-zval = 34.92 2024/11/23-20:15:15.197986 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278877.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:15:16.045287 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278877.sst): second pass time = 1.32's, 81.843'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.418'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3318.626'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 303.916'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35524.527'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.843'MB/sec zip pipeline throughput = 81.843'MB/sec entries = 928686 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.52 avg-zval = 34.92 usrkeys = 928686 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.52 avg-zval = 34.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928686 bytes = 3250416 } seq expand size = 6500802 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.4454 value = 3.3370 dict = 1.00 all = 3.4823 } Zip/UnZip{ index = 0.0957 value = 0.2997 dict = 1.00 all = 0.2872 } ---------------------------- total value len = 11.015727 GB avg = 0.157 KB (by entry num) total key len = 1.934810 GB avg = 0.028 KB total ukey len = 1.350728 GB avg = 0.016 KB total ukey num = 0.083192910 Billion total entry num = 0.070000639 Billion write speed all = 0.317385322 MB/sec (with seq num) write speed all = 0.303660995 MB/sec (without seq num) 2024/11/23-20:15:16.093617 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278877: 928686 keys, 38542824 bytes, temperature: kWarm 2024/11/23-20:15:16.093685 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364116093649, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278877, "file_size": 38542824, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4238004906, "table_properties": {"data_size": 32428464, "index_size": 1778176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003208, "raw_average_key_size": 28, "raw_value_size": 108214536, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928686, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364112, "oldest_key_time": 0, "file_creation_time": 1732364112, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278877, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:15:16.130217 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278878.sst) /dev/shm/ToplingTemp/Topling-176930-TeVCVQ 2024/11/23-20:15:16.264691 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278878.sst): first pass time = 0.13's, 229.319'MB/sec 2024/11/23-20:15:16.557763 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278878.sst): old prealloc_size = 147639500, real_size = 8748080 2024/11/23-20:15:16.557936 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278878.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 214529 prefix = 4 raw-key = 0.0034 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0248 GB zip-val = 0.0083 GB avg-val = 115.69 avg-zval = 38.62 2024/11/23-20:15:16.558168 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278878.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.031 GB 2024/11/23-20:15:16.753261 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00/278878.sst): second pass time = 0.29's, 84.894'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 49.100'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2745.807'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.328'MB/sec write SST data time = 0.03's, 302.735'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.257'MB, throughput = 8293.554'MB/sec zip my value time = 0.29's, unzip length = 0.025'GB zip my value throughput = 84.894'MB/sec zip pipeline throughput = 84.894'MB/sec entries = 214529 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.69 avg-zval = 38.62 usrkeys = 214529 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.69 avg-zval = 38.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 214529 bytes = 643600 } seq expand size = 1501703 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0043 GB value = 0.0248 GB dict = 0.26 MB all = 0.0308 GB } __ZipSize{ index = 0.0004 GB value = 0.0083 GB dict = 0.26 MB all = 0.0096 GB } UnZip/Zip{ index = 10.4800 value = 2.9957 dict = 1.00 all = 3.2118 } Zip/UnZip{ index = 0.0954 value = 0.3338 dict = 1.00 all = 0.3114 } ---------------------------- total value len = 11.042047 GB avg = 0.157 KB (by entry num) total key len = 1.940817 GB avg = 0.028 KB total ukey len = 1.355018 GB avg = 0.016 KB total ukey num = 0.083407439 Billion total entry num = 0.070215168 Billion write speed all = 0.318172058 MB/sec (with seq num) write speed all = 0.304405910 MB/sec (without seq num) 2024/11/23-20:15:16.770896 139957676762688 [compaction_job.cc:1948] [default] [JOB 34971] Generated table #278878: 214529 keys, 9597511 bytes, temperature: kWarm 2024/11/23-20:15:16.770953 139957676762688 EVENT_LOG_v1 {"time_micros": 1732364116770924, "cf_name": "default", "job": 34971, "event": "table_file_creation", "file_number": 278878, "file_size": 9597511, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4220739721, "largest_seqno": 4236508540, "table_properties": {"data_size": 8284832, "index_size": 409408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6006812, "raw_average_key_size": 28, "raw_value_size": 24818610, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 214529, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364116, "oldest_key_time": 0, "file_creation_time": 1732364116, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278878, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:15:16.784325 139957676762688 [compaction_job.cc:703] job-34971: subcompact[0], size: 0.730826 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34971/att-00 [278859,278860,278861,278862,278863,278864,278865,278866,278867,278868,278869,278870,278871,278872,278873,278874,278875,278876,278877,278878] 2024/11/23-20:15:16.906911 139957676762688 (Original Log Time 2024/11/23-20:14:07.939878) EVENT_LOG_v1 {"time_micros": 1732364047939819, "job": 34971, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [278848], "files_L3": [278807, 278808, 278809, 278810, 278811, 278812, 278784, 278785, 278786, 278787, 278788, 278789, 278790, 278791, 278792, 278793, 278794, 278795, 278796, 278797], "score": 0.989399, "input_data_size": 755466559, "oldest_snapshot_seqno": 4238082390} 2024/11/23-20:15:16.906915 139957676762688 (Original Log Time 2024/11/23-20:15:16.905381) [compaction_job.cc:2022] [default] [JOB 34971] Compacted 1@2 + 20@3 files to L3 => 730826205 bytes 2024/11/23-20:15:16.906917 139957676762688 (Original Log Time 2024/11/23-20:15:16.906842) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.0 rd, 10.6 wr, level 3, files in(1, 20) out(20 +0 blob) MB in(27.1, 693.3 +0.0 blob) out(697.0 +0.0 blob), read-write-amplify(52.3) write-amplify(25.7) OK, records in: 17973818, records dropped: 116814 output_compression: Snappy 2024/11/23-20:15:16.906920 139957676762688 (Original Log Time 2024/11/23-20:15:16.906885) EVENT_LOG_v1 {"time_micros": 1732364116906865, "cf": "default", "job": 34971, "event": "compaction_finished", "compaction_time_micros": 68844453, "compaction_time_cpu_micros": 29851855, "output_level": 3, "num_output_files": 20, "total_output_size": 730826205, "num_input_records": 17973818, "num_output_records": 17857004, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 571063608, "file_range_sync_nanos": 0, "file_fsync_nanos": 15382, "file_prepare_write_nanos": 61431, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-20:15:17.045378 139957676762688 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732364047,"executesMs":69098,"compactionJobId":34971,"attempt":0,"compactionInputRawBytes":2597551392,"compactionInputZipBytes":755466559,"compactionOutputRawBytes":0,"compactionOutputZipBytes":730826205,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-92f33eb9601eb4ecd328a4c1a3f39cfa-02ac7e0b91a7aa44-00"}}, response = 2024/11/23-20:15:17.045472 139957676762688 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34971/att-00: olev 3, work 69.098 s, result 7.860 ms, install 1.554 ms, input{raw 2.419 GiB zip 720.469 MiB}