2024/11/22-19:32:04.647369 140024995214912 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00: fork to child time = 0.009230 sec 2024/11/22-19:32:04.648581 140024995214912 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00 2024/11/22-19:32:04.655860 140024995214912 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00 2024/11/22-19:32:04.657031 140024995214912 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-19:32:04.657321 140024995214912 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-19:32:04.703162 140024995214912 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08938/att-00/MANIFEST-206347 2024/11/22-19:32:04.737763 140024995214912 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08938/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 223473, last_sequence is 3505688639, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 223264 2024/11/22-19:32:04.737773 140024995214912 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-19:32:04.737888 140024995214912 [version_set.cc:5721] Creating manifest 223473 2024/11/22-19:32:04.768889 140024995214912 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-19:32:04.790461 140024995214912 [compaction_job.cc:2437] [default] [JOB 8938] Compacting 1@2 + 11@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-19:32:04.790482 140024995214912 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 6772 Base level 2, inputs: [223399(21MB)], [223470(35MB) 223471(6843KB) 223364(34MB) 223365(35MB) 223366(35MB) 223367(34MB) 223368(35MB) 223369(35MB) 223370(36MB) 223371(37MB) 223448(36MB)] 2024/11/22-19:32:04.797152 140024995214912 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:32:04.797186 140024995214912 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:32:04.797208 140024995214912 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:32:04.797213 140024995214912 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-19:32:04.797225 140024995214912 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-19:32:05.312922 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223474.sst) /dev/shm/ToplingTemp/Topling-260603-RM55iK 2024/11/22-19:32:05.863831 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223474.sst): first pass time = 0.55's, 243.656'MB/sec 2024/11/22-19:32:07.192060 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223474.sst): old prealloc_size = 147639500, real_size = 33690544 2024/11/22-19:32:07.192600 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223474.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923305 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.37 avg-zval = 34.34 2024/11/22-19:32:07.193404 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223474.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:32:08.130397 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223474.sst): second pass time = 1.33's, 81.666'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.140'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3373.331'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.285'MB/sec write SST data time = 0.13's, 291.260'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33813.503'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.666'MB/sec zip pipeline throughput = 81.666'MB/sec entries = 923305 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.37 avg-zval = 34.34 usrkeys = 923305 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.37 avg-zval = 34.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923305 bytes = 3116176 } seq expand size = 6463135 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5139 value = 3.4181 dict = 1.00 all = 3.5635 } Zip/UnZip{ index = 0.0951 value = 0.2926 dict = 1.00 all = 0.2806 } ---------------------------- total value len = 0.114828 GB avg = 0.124 KB (by entry num) total key len = 0.025853 GB avg = 0.028 KB total ukey len = 0.018466 GB avg = 0.020 KB total ukey num = 0.000923305 Billion total entry num = 0.000923305 Billion write speed all = 0.003664995 MB/sec (with seq num) write speed all = 0.003472564 MB/sec (without seq num) 2024/11/22-19:32:08.166623 140024995214912 [compaction_job.cc:1948] [default] [JOB 8938] Generated table #223474: 923305 keys, 37664552 bytes, temperature: kWarm 2024/11/22-19:32:08.166711 140024995214912 EVENT_LOG_v1 {"time_micros": 1732275128166657, "cf_name": "default", "job": 8938, "event": "table_file_creation", "file_number": 223474, "file_size": 37664552, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3505103988, "table_properties": {"data_size": 31703200, "index_size": 1756352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25852540, "raw_average_key_size": 28, "raw_value_size": 108365270, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923305, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275125, "oldest_key_time": 0, "file_creation_time": 1732275125, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223474, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:32:08.204079 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223475.sst) /dev/shm/ToplingTemp/Topling-260603-Er02NK 2024/11/22-19:32:09.088969 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223475.sst): first pass time = 0.88's, 151.684'MB/sec 2024/11/22-19:32:10.414475 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223475.sst): old prealloc_size = 147639500, real_size = 33382736 2024/11/22-19:32:10.415043 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223475.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923488 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0314 GB avg-val = 117.34 avg-zval = 34.00 2024/11/22-19:32:10.415807 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223475.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:32:11.346422 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223475.sst): second pass time = 1.32's, 81.797'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.086'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3224.201'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 289.660'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35177.116'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.797'MB/sec zip pipeline throughput = 81.797'MB/sec entries = 923488 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.34 avg-zval = 34.00 usrkeys = 923488 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.34 avg-zval = 34.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923488 bytes = 3116784 } seq expand size = 6464416 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5129 value = 3.4515 dict = 1.00 all = 3.5928 } Zip/UnZip{ index = 0.0951 value = 0.2897 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 0.229653 GB avg = 0.124 KB (by entry num) total key len = 0.051710 GB avg = 0.028 KB total ukey len = 0.036936 GB avg = 0.020 KB total ukey num = 0.001846793 Billion total entry num = 0.001846793 Billion write speed all = 0.007329407 MB/sec (with seq num) write speed all = 0.006944540 MB/sec (without seq num) 2024/11/22-19:32:11.380045 140024995214912 [compaction_job.cc:1948] [default] [JOB 8938] Generated table #223475: 923488 keys, 37357270 bytes, temperature: kWarm 2024/11/22-19:32:11.380111 140024995214912 EVENT_LOG_v1 {"time_micros": 1732275131380077, "cf_name": "default", "job": 8938, "event": "table_file_creation", "file_number": 223475, "file_size": 37357270, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774766, "largest_seqno": 3505104570, "table_properties": {"data_size": 31394816, "index_size": 1756864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25857664, "raw_average_key_size": 28, "raw_value_size": 108360068, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923488, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275128, "oldest_key_time": 0, "file_creation_time": 1732275128, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223475, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:32:11.420042 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223476.sst) /dev/shm/ToplingTemp/Topling-260603-qZoPxM 2024/11/22-19:32:12.234249 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223476.sst): first pass time = 0.81's, 164.854'MB/sec 2024/11/22-19:32:13.555881 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223476.sst): old prealloc_size = 147639500, real_size = 32330992 2024/11/22-19:32:13.556406 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223476.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927244 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 116.75 avg-zval = 32.72 2024/11/22-19:32:13.557183 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223476.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:32:14.465238 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223476.sst): second pass time = 1.32's, 81.950'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 39.935'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3496.000'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 286.357'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33337.696'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.950'MB/sec zip pipeline throughput = 81.950'MB/sec entries = 927244 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 32.72 usrkeys = 927244 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 32.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927244 bytes = 3129456 } seq expand size = 6490708 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5151 value = 3.5686 dict = 1.00 all = 3.6960 } Zip/UnZip{ index = 0.0951 value = 0.2802 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 0.344399 GB avg = 0.124 KB (by entry num) total key len = 0.077673 GB avg = 0.028 KB total ukey len = 0.055481 GB avg = 0.020 KB total ukey num = 0.002774037 Billion total entry num = 0.002774037 Billion write speed all = 0.010993919 MB/sec (with seq num) write speed all = 0.010415864 MB/sec (without seq num) 2024/11/22-19:32:14.498185 140024995214912 [compaction_job.cc:1948] [default] [JOB 8938] Generated table #223476: 927244 keys, 36314556 bytes, temperature: kWarm 2024/11/22-19:32:14.498244 140024995214912 EVENT_LOG_v1 {"time_micros": 1732275134498213, "cf_name": "default", "job": 8938, "event": "table_file_creation", "file_number": 223476, "file_size": 36314556, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774786, "largest_seqno": 3505104885, "table_properties": {"data_size": 30335328, "index_size": 1763648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962832, "raw_average_key_size": 28, "raw_value_size": 108254956, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927244, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275131, "oldest_key_time": 0, "file_creation_time": 1732275131, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223476, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:32:14.532992 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223477.sst) /dev/shm/ToplingTemp/Topling-260603-Nba7fL 2024/11/22-19:32:15.356991 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223477.sst): first pass time = 0.82's, 162.894'MB/sec 2024/11/22-19:32:16.685316 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223477.sst): old prealloc_size = 147639500, real_size = 32591040 2024/11/22-19:32:16.685877 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223477.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924486 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 117.18 avg-zval = 33.10 2024/11/22-19:32:16.686550 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223477.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:32:17.553587 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223477.sst): second pass time = 1.33's, 81.637'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.676'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3262.966'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.13's, 279.993'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 24457.117'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.637'MB/sec zip pipeline throughput = 81.637'MB/sec entries = 924486 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.18 avg-zval = 33.10 usrkeys = 924486 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.99 avg-zval = 33.10 TagRS{ kinds = RS_Key0_TagN bytes = 144560 } TagArray{ size = 767491 bytes = 2590304 } seq expand size = 5372437 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5139 value = 3.5401 dict = 1.00 all = 3.7092 } Zip/UnZip{ index = 0.0951 value = 0.2825 dict = 1.00 all = 0.2696 } ---------------------------- total value len = 0.458103 GB avg = 0.124 KB (by entry num) total key len = 0.103559 GB avg = 0.028 KB total ukey len = 0.073970 GB avg = 0.020 KB total ukey num = 0.003698523 Billion total entry num = 0.003698523 Billion write speed all = 0.014628721 MB/sec (with seq num) write speed all = 0.013858085 MB/sec (without seq num) 2024/11/22-19:32:17.584736 140024995214912 [compaction_job.cc:1948] [default] [JOB 8938] Generated table #223477: 924486 keys, 36185440 bytes, temperature: kWarm 2024/11/22-19:32:17.584818 140024995214912 EVENT_LOG_v1 {"time_micros": 1732275137584776, "cf_name": "default", "job": 8938, "event": "table_file_creation", "file_number": 223477, "file_size": 36185440, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3505104051, "table_properties": {"data_size": 30601136, "index_size": 1758592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25885608, "raw_average_key_size": 28, "raw_value_size": 108332192, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924486, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275134, "oldest_key_time": 0, "file_creation_time": 1732275134, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223477, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:32:17.621984 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223478.sst) /dev/shm/ToplingTemp/Topling-260603-wNpzJN 2024/11/22-19:32:18.523815 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223478.sst): first pass time = 0.90's, 148.835'MB/sec 2024/11/22-19:32:19.920397 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223478.sst): old prealloc_size = 147639500, real_size = 35012256 2024/11/22-19:32:19.921050 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223478.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081669 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 96.08 avg-zval = 30.22 2024/11/22-19:32:19.921103 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223478.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:32:20.547802 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223478.sst): second pass time = 1.40's, 74.453'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.042'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3267.206'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.555'MB/sec write SST data time = 0.14's, 256.019'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.034'MB, throughput = 27565.186'MB/sec zip my value time = 1.40's, unzip length = 0.104'GB zip my value throughput = 74.453'MB/sec zip pipeline throughput = 74.453'MB/sec entries = 1081669 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.08 avg-zval = 30.22 usrkeys = 1081669 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.08 avg-zval = 30.22 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.03 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5159 value = 3.1798 dict = 1.00 all = 3.7505 } Zip/UnZip{ index = 0.0951 value = 0.3145 dict = 1.00 all = 0.2666 } ---------------------------- total value len = 0.562034 GB avg = 0.118 KB (by entry num) total key len = 0.133845 GB avg = 0.028 KB total ukey len = 0.095604 GB avg = 0.020 KB total ukey num = 0.004780192 Billion total entry num = 0.004780192 Billion write speed all = 0.018123068 MB/sec (with seq num) write speed all = 0.017127128 MB/sec (without seq num) 2024/11/22-19:32:20.578808 140024995214912 [compaction_job.cc:1948] [default] [JOB 8938] Generated table #223478: 1081669 keys, 35786621 bytes, temperature: kWarm 2024/11/22-19:32:20.578873 140024995214912 EVENT_LOG_v1 {"time_micros": 1732275140578841, "cf_name": "default", "job": 8938, "event": "table_file_creation", "file_number": 223478, "file_size": 35786621, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32684432, "index_size": 2057216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30286732, "raw_average_key_size": 28, "raw_value_size": 103931063, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081669, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275137, "oldest_key_time": 0, "file_creation_time": 1732275137, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223478, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:32:20.619619 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223479.sst) /dev/shm/ToplingTemp/Topling-260603-EwQRkO 2024/11/22-19:32:21.408428 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223479.sst): first pass time = 0.79's, 170.163'MB/sec 2024/11/22-19:32:22.729043 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223479.sst): old prealloc_size = 147639500, real_size = 32762528 2024/11/22-19:32:22.729583 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223479.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924107 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.24 avg-zval = 33.30 2024/11/22-19:32:22.730301 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223479.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:32:23.608962 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223479.sst): second pass time = 1.32's, 82.114'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.456'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3397.905'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 280.531'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34081.202'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.114'MB/sec zip pipeline throughput = 82.114'MB/sec entries = 924107 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.24 avg-zval = 33.30 usrkeys = 924107 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.11 avg-zval = 33.30 TagRS{ kinds = RS_Key0_TagN bytes = 144480 } TagArray{ size = 774368 bytes = 2710304 } seq expand size = 5420576 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5123 value = 3.5207 dict = 1.00 all = 3.6795 } Zip/UnZip{ index = 0.0951 value = 0.2840 dict = 1.00 all = 0.2718 } ---------------------------- total value len = 0.675798 GB avg = 0.118 KB (by entry num) total key len = 0.159720 GB avg = 0.028 KB total ukey len = 0.114086 GB avg = 0.020 KB total ukey num = 0.005704299 Billion total entry num = 0.005704299 Billion write speed all = 0.021757990 MB/sec (with seq num) write speed all = 0.020569610 MB/sec (without seq num) 2024/11/22-19:32:23.642468 140024995214912 [compaction_job.cc:1948] [default] [JOB 8938] Generated table #223479: 924107 keys, 36477674 bytes, temperature: kWarm 2024/11/22-19:32:23.642529 140024995214912 EVENT_LOG_v1 {"time_micros": 1732275143642498, "cf_name": "default", "job": 8938, "event": "table_file_creation", "file_number": 223479, "file_size": 36477674, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3505105086, "table_properties": {"data_size": 30773200, "index_size": 1758144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25874996, "raw_average_key_size": 28, "raw_value_size": 108342790, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924107, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275140, "oldest_key_time": 0, "file_creation_time": 1732275140, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223479, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:32:23.681075 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223480.sst) /dev/shm/ToplingTemp/Topling-260603-iaYsHM 2024/11/22-19:32:24.588103 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223480.sst): first pass time = 0.91's, 147.982'MB/sec 2024/11/22-19:32:25.925062 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223480.sst): old prealloc_size = 147639500, real_size = 32986896 2024/11/22-19:32:25.925630 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223480.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923619 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0310 GB avg-val = 117.32 avg-zval = 33.56 2024/11/22-19:32:25.926410 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223480.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:32:26.818401 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223480.sst): second pass time = 1.34's, 81.087'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.505'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3140.281'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.302'MB/sec write SST data time = 0.13's, 288.211'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34670.638'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.087'MB/sec zip pipeline throughput = 81.087'MB/sec entries = 923619 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.32 avg-zval = 33.56 usrkeys = 923619 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.32 avg-zval = 33.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923619 bytes = 3232688 } seq expand size = 6465333 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5121 value = 3.4955 dict = 1.00 all = 3.6200 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 0.790619 GB avg = 0.119 KB (by entry num) total key len = 0.185582 GB avg = 0.028 KB total ukey len = 0.132558 GB avg = 0.020 KB total ukey num = 0.006627918 Billion total entry num = 0.006627918 Billion write speed all = 0.025419441 MB/sec (with seq num) write speed all = 0.024038758 MB/sec (without seq num) 2024/11/22-19:32:26.851318 140024995214912 [compaction_job.cc:1948] [default] [JOB 8938] Generated table #223480: 923619 keys, 37077004 bytes, temperature: kWarm 2024/11/22-19:32:26.851381 140024995214912 EVENT_LOG_v1 {"time_micros": 1732275146851349, "cf_name": "default", "job": 8938, "event": "table_file_creation", "file_number": 223480, "file_size": 37077004, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3365258144, "largest_seqno": 3505105098, "table_properties": {"data_size": 30998528, "index_size": 1757248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25861332, "raw_average_key_size": 28, "raw_value_size": 108356498, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923619, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275143, "oldest_key_time": 0, "file_creation_time": 1732275143, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223480, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:32:26.893666 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223481.sst) /dev/shm/ToplingTemp/Topling-260603-LUNqjK 2024/11/22-19:32:27.731348 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223481.sst): first pass time = 0.84's, 160.233'MB/sec 2024/11/22-19:32:29.056829 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223481.sst): old prealloc_size = 147639500, real_size = 33308080 2024/11/22-19:32:29.057352 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223481.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923318 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0313 GB avg-val = 117.36 avg-zval = 33.92 2024/11/22-19:32:29.058132 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223481.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:32:29.957721 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223481.sst): second pass time = 1.32's, 81.795'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 41.512'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3516.341'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.299'MB/sec write SST data time = 0.13's, 289.799'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31855.134'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.795'MB/sec zip pipeline throughput = 81.795'MB/sec entries = 923318 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.36 avg-zval = 33.92 usrkeys = 923318 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.36 avg-zval = 33.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923318 bytes = 3231632 } seq expand size = 6463226 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5137 value = 3.4599 dict = 1.00 all = 3.5889 } Zip/UnZip{ index = 0.0951 value = 0.2890 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 0.905448 GB avg = 0.120 KB (by entry num) total key len = 0.211435 GB avg = 0.028 KB total ukey len = 0.151025 GB avg = 0.020 KB total ukey num = 0.007551236 Billion total entry num = 0.007551236 Billion write speed all = 0.029080277 MB/sec (with seq num) write speed all = 0.027507384 MB/sec (without seq num) 2024/11/22-19:32:29.991496 140024995214912 [compaction_job.cc:1948] [default] [JOB 8938] Generated table #223481: 923318 keys, 37397524 bytes, temperature: kWarm 2024/11/22-19:32:29.991561 140024995214912 EVENT_LOG_v1 {"time_micros": 1732275149991528, "cf_name": "default", "job": 8938, "event": "table_file_creation", "file_number": 223481, "file_size": 37397524, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3505105112, "table_properties": {"data_size": 31320672, "index_size": 1756416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25852904, "raw_average_key_size": 28, "raw_value_size": 108364900, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923318, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275146, "oldest_key_time": 0, "file_creation_time": 1732275146, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223481, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:32:30.028376 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223482.sst) /dev/shm/ToplingTemp/Topling-260603-0m8qdO 2024/11/22-19:32:30.903968 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223482.sst): first pass time = 0.88's, 153.297'MB/sec 2024/11/22-19:32:32.234534 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223482.sst): old prealloc_size = 147639500, real_size = 33315808 2024/11/22-19:32:32.235089 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223482.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924733 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 117.14 avg-zval = 33.87 2024/11/22-19:32:32.235862 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223482.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:32:33.149466 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223482.sst): second pass time = 1.33's, 81.454'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.887'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3412.807'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 290.921'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 25081.174'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.454'MB/sec zip pipeline throughput = 81.454'MB/sec entries = 924733 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.14 avg-zval = 33.87 usrkeys = 924733 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.14 avg-zval = 33.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924733 bytes = 3236576 } seq expand size = 6473131 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5122 value = 3.4581 dict = 1.00 all = 3.5879 } Zip/UnZip{ index = 0.0951 value = 0.2892 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 1.020246 GB avg = 0.120 KB (by entry num) total key len = 0.237327 GB avg = 0.028 KB total ukey len = 0.169519 GB avg = 0.020 KB total ukey num = 0.008475969 Billion total entry num = 0.008475969 Billion write speed all = 0.032740730 MB/sec (with seq num) write speed all = 0.030975365 MB/sec (without seq num) 2024/11/22-19:32:33.182755 140024995214912 [compaction_job.cc:1948] [default] [JOB 8938] Generated table #223482: 924733 keys, 37408116 bytes, temperature: kWarm 2024/11/22-19:32:33.182828 140024995214912 EVENT_LOG_v1 {"time_micros": 1732275153182795, "cf_name": "default", "job": 8938, "event": "table_file_creation", "file_number": 223482, "file_size": 37408116, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3319401908, "largest_seqno": 3505105139, "table_properties": {"data_size": 31325072, "index_size": 1759360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25892524, "raw_average_key_size": 28, "raw_value_size": 108325342, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924733, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275150, "oldest_key_time": 0, "file_creation_time": 1732275150, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223482, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:32:33.225560 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223483.sst) /dev/shm/ToplingTemp/Topling-260603-To6JyN 2024/11/22-19:32:34.159738 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223483.sst): first pass time = 0.93's, 143.681'MB/sec 2024/11/22-19:32:35.479711 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223483.sst): old prealloc_size = 147639500, real_size = 32371936 2024/11/22-19:32:35.480295 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223483.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923200 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0304 GB avg-val = 117.38 avg-zval = 32.91 2024/11/22-19:32:35.481071 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223483.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:32:36.365049 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223483.sst): second pass time = 1.32's, 82.142'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.184'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3135.926'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 282.744'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32386.030'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.142'MB/sec zip pipeline throughput = 82.142'MB/sec entries = 923200 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.38 avg-zval = 32.91 usrkeys = 923200 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.38 avg-zval = 32.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923200 bytes = 3231216 } seq expand size = 6462400 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5131 value = 3.5665 dict = 1.00 all = 3.6811 } Zip/UnZip{ index = 0.0951 value = 0.2804 dict = 1.00 all = 0.2717 } ---------------------------- total value len = 1.135077 GB avg = 0.121 KB (by entry num) total key len = 0.263177 GB avg = 0.028 KB total ukey len = 0.187983 GB avg = 0.020 KB total ukey num = 0.009399169 Billion total entry num = 0.009399169 Billion write speed all = 0.036400272 MB/sec (with seq num) write speed all = 0.034442788 MB/sec (without seq num) 2024/11/22-19:32:36.397990 140024995214912 [compaction_job.cc:1948] [default] [JOB 8938] Generated table #223483: 923200 keys, 36461214 bytes, temperature: kWarm 2024/11/22-19:32:36.398059 140024995214912 EVENT_LOG_v1 {"time_micros": 1732275156398022, "cf_name": "default", "job": 8938, "event": "table_file_creation", "file_number": 223483, "file_size": 36461214, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3505105153, "table_properties": {"data_size": 30384656, "index_size": 1756288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25849600, "raw_average_key_size": 28, "raw_value_size": 108368248, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923200, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275153, "oldest_key_time": 0, "file_creation_time": 1732275153, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223483, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:32:36.435550 140024995214912 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223484.sst) /dev/shm/ToplingTemp/Topling-260603-JS9hZN 2024/11/22-19:32:36.597553 140024995214912 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223484.sst): first pass time = 0.16's, 238.528'MB/sec 2024/11/22-19:32:36.962867 140024995214912 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223484.sst): old prealloc_size = 147639500, real_size = 11125232 2024/11/22-19:32:36.963059 140024995214912 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223484.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 266705 prefix = 4 raw-key = 0.0043 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0312 GB zip-val = 0.0105 GB avg-val = 116.85 avg-zval = 39.56 2024/11/22-19:32:36.963314 140024995214912 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223484.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.039 GB 2024/11/22-19:32:37.208337 140024995214912 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00/223484.sst): second pass time = 0.36's, 85.443'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 50.160'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3003.951'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.321'MB/sec write SST data time = 0.04's, 328.917'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.318'MB, throughput = 10522.246'MB/sec zip my value time = 0.36's, unzip length = 0.031'GB zip my value throughput = 85.443'MB/sec zip pipeline throughput = 85.443'MB/sec entries = 266705 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.85 avg-zval = 39.56 usrkeys = 266705 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.85 avg-zval = 39.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 266705 bytes = 933488 } seq expand size = 1866935 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0053 GB value = 0.0312 GB dict = 0.32 MB all = 0.0386 GB } __ZipSize{ index = 0.0005 GB value = 0.0105 GB dict = 0.32 MB all = 0.0123 GB } UnZip/Zip{ index = 10.4876 value = 2.9541 dict = 1.00 all = 3.1377 } Zip/UnZip{ index = 0.0954 value = 0.3385 dict = 1.00 all = 0.3187 } ---------------------------- total value len = 1.168108 GB avg = 0.121 KB (by entry num) total key len = 0.270644 GB avg = 0.028 KB total ukey len = 0.193317 GB avg = 0.020 KB total ukey num = 0.009665874 Billion total entry num = 0.009665874 Billion write speed all = 0.037453756 MB/sec (with seq num) write speed all = 0.035440772 MB/sec (without seq num) 2024/11/22-19:32:37.224470 140024995214912 [compaction_job.cc:1948] [default] [JOB 8938] Generated table #223484: 266705 keys, 12312189 bytes, temperature: kWarm 2024/11/22-19:32:37.224537 140024995214912 EVENT_LOG_v1 {"time_micros": 1732275157224505, "cf_name": "default", "job": 8938, "event": "table_file_creation", "file_number": 223484, "file_size": 12312189, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701165, "largest_seqno": 3505105163, "table_properties": {"data_size": 10549792, "index_size": 508608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7467740, "raw_average_key_size": 28, "raw_value_size": 31164706, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 266705, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275156, "oldest_key_time": 0, "file_creation_time": 1732275156, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223484, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:32:37.241313 140024995214912 [compaction_job.cc:703] job-08938: subcompact[0], size: 0.380442 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08938/att-00 [223474,223475,223476,223477,223478,223479,223480,223481,223482,223483,223484] 2024/11/22-19:32:37.309732 140024995214912 (Original Log Time 2024/11/22-19:32:04.790547) EVENT_LOG_v1 {"time_micros": 1732275124790499, "job": 8938, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [223399], "files_L3": [223470, 223471, 223364, 223365, 223366, 223367, 223368, 223369, 223370, 223371, 223448], "score": 0.996598, "input_data_size": 403777073, "oldest_snapshot_seqno": 3505688621} 2024/11/22-19:32:37.309736 140024995214912 (Original Log Time 2024/11/22-19:32:37.309069) [compaction_job.cc:2022] [default] [JOB 8938] Compacted 1@2 + 11@3 files to L3 => 380442160 bytes 2024/11/22-19:32:37.309737 140024995214912 (Original Log Time 2024/11/22-19:32:37.309665) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 12.4 rd, 11.7 wr, level 3, files in(1, 11) out(11 +0 blob) MB in(21.0, 364.0 +0.0 blob) out(362.8 +0.0 blob), read-write-amplify(35.6) write-amplify(17.3) OK, records in: 9732507, records dropped: 66633 output_compression: Snappy 2024/11/22-19:32:37.309744 140024995214912 (Original Log Time 2024/11/22-19:32:37.309711) EVENT_LOG_v1 {"time_micros": 1732275157309689, "cf": "default", "job": 8938, "event": "compaction_finished", "compaction_time_micros": 32450774, "compaction_time_cpu_micros": 14697521, "output_level": 3, "num_output_files": 11, "total_output_size": 380442160, "num_input_records": 9732507, "num_output_records": 9665874, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 241554912, "file_range_sync_nanos": 0, "file_fsync_nanos": 10771, "file_prepare_write_nanos": 25631, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/22-19:32:37.432061 140024995214912 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732275124,"executesMs":32655,"compactionJobId":8938,"attempt":0,"compactionInputRawBytes":1390272391,"compactionInputZipBytes":403777073,"compactionOutputRawBytes":0,"compactionOutputZipBytes":380442160,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-b98e4e4c62ce56c681127914451a252b-ccd4a350dfe25f86-00"}}, response = 2024/11/22-19:32:37.432132 140024995214912 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08938/att-00: olev 3, work 32.655 s, result 6.311 ms, install 0.695 ms, input{raw 1.295 GiB zip 385.072 MiB}