2024/11/23-06:32:10.890036 140024743433792 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00: fork to child time = 0.030712 sec 2024/11/23-06:32:10.891040 140024743433792 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00 2024/11/23-06:32:10.900488 140024743433792 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00 2024/11/23-06:32:10.902002 140024743433792 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-06:32:10.902384 140024743433792 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-06:32:10.954097 140024743433792 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21104/att-00/MANIFEST-206347 2024/11/23-06:32:10.983762 140024743433792 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21104/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 248717, last_sequence is 3837555337, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 248530 2024/11/23-06:32:10.983771 140024743433792 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-06:32:10.983879 140024743433792 [version_set.cc:5721] Creating manifest 248717 2024/11/23-06:32:11.025242 140024743433792 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-06:32:11.044287 140024743433792 [compaction_job.cc:2437] [default] [JOB 21104] Compacting 1@2 + 15@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-06:32:11.044303 140024743433792 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 15799 Base level 2, inputs: [248677(27MB)], [248605(35MB) 248606(33MB) 248607(34MB) 248609(35MB) 248610(34MB) 248611(35MB) 248612(34MB) 248634(35MB) 248635(35MB) 248636(36MB) 248637(33MB) 248638(33MB) 248639(35MB) 248640(35MB) 248702(34MB)] 2024/11/23-06:32:11.050170 140024743433792 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:32:11.050203 140024743433792 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:32:11.050223 140024743433792 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:32:11.050228 140024743433792 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:32:11.050239 140024743433792 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:32:11.698521 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248718.sst) /dev/shm/ToplingTemp/Topling-435562-jFFPWm 2024/11/23-06:32:12.233203 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248718.sst): first pass time = 0.53's, 251.046'MB/sec 2024/11/23-06:32:13.615068 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248718.sst): old prealloc_size = 147639500, real_size = 32550896 2024/11/23-06:32:13.615581 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248718.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928861 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.50 avg-zval = 32.89 2024/11/23-06:32:13.616417 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248718.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:14.608820 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248718.sst): second pass time = 1.38's, 78.373'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 37.003'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3585.476'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.269'MB/sec write SST data time = 0.13's, 279.852'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32174.431'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.373'MB/sec zip pipeline throughput = 78.373'MB/sec entries = 928861 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.50 avg-zval = 32.89 usrkeys = 928861 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.50 avg-zval = 32.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928861 bytes = 3367136 } seq expand size = 6502027 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5139 value = 3.5419 dict = 1.00 all = 3.6501 } Zip/UnZip{ index = 0.0951 value = 0.2823 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 0.114712 GB avg = 0.123 KB (by entry num) total key len = 0.026008 GB avg = 0.028 KB total ukey len = 0.018577 GB avg = 0.020 KB total ukey num = 0.000928861 Billion total entry num = 0.000928861 Billion write speed all = 0.001804297 MB/sec (with seq num) write speed all = 0.001709019 MB/sec (without seq num) 2024/11/23-06:32:14.642498 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248718: 928861 keys, 36771084 bytes, temperature: kWarm 2024/11/23-06:32:14.642584 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314734642531, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248718, "file_size": 36771084, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3837397211, "table_properties": {"data_size": 30551584, "index_size": 1766912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26008108, "raw_average_key_size": 28, "raw_value_size": 108209718, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928861, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314731, "oldest_key_time": 0, "file_creation_time": 1732314731, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248718, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:14.677972 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248719.sst) /dev/shm/ToplingTemp/Topling-435562-RJxBMk 2024/11/23-06:32:15.551969 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248719.sst): first pass time = 0.87's, 153.574'MB/sec 2024/11/23-06:32:16.886201 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248719.sst): old prealloc_size = 147639500, real_size = 30896800 2024/11/23-06:32:16.886733 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248719.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925517 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.02 avg-zval = 31.21 2024/11/23-06:32:16.887518 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248719.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:17.830033 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248719.sst): second pass time = 1.33's, 81.211'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 37.072'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3503.234'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.13's, 274.706'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33634.293'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.211'MB/sec zip pipeline throughput = 81.211'MB/sec entries = 925517 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.02 avg-zval = 31.21 usrkeys = 925517 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.02 avg-zval = 31.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925517 bytes = 3239328 } seq expand size = 6478619 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.3754 value = 3.7500 dict = 1.00 all = 3.8359 } Zip/UnZip{ index = 0.0964 value = 0.2667 dict = 1.00 all = 0.2607 } ---------------------------- total value len = 0.229494 GB avg = 0.124 KB (by entry num) total key len = 0.051923 GB avg = 0.028 KB total ukey len = 0.037088 GB avg = 0.020 KB total ukey num = 0.001854378 Billion total entry num = 0.001854378 Billion write speed all = 0.003608145 MB/sec (with seq num) write speed all = 0.003417940 MB/sec (without seq num) 2024/11/23-06:32:17.863603 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248719: 925517 keys, 34989526 bytes, temperature: kWarm 2024/11/23-06:32:17.863666 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314737863635, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248719, "file_size": 34989526, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3666772762, "largest_seqno": 3837396679, "table_properties": {"data_size": 28881168, "index_size": 1784064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25914476, "raw_average_key_size": 28, "raw_value_size": 108303378, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925517, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314734, "oldest_key_time": 0, "file_creation_time": 1732314734, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248719, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:17.918019 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248720.sst) /dev/shm/ToplingTemp/Topling-435562-MpeFSn 2024/11/23-06:32:18.749523 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248720.sst): first pass time = 0.83's, 161.423'MB/sec 2024/11/23-06:32:20.126009 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248720.sst): old prealloc_size = 147639500, real_size = 32253280 2024/11/23-06:32:20.126525 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248720.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930890 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0302 GB avg-val = 116.18 avg-zval = 32.48 2024/11/23-06:32:20.127339 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248720.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:21.034945 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248720.sst): second pass time = 1.38's, 78.637'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.134'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3599.520'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 283.439'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33379.795'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.637'MB/sec zip pipeline throughput = 78.637'MB/sec entries = 930890 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.18 avg-zval = 32.48 usrkeys = 930890 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.18 avg-zval = 32.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930890 bytes = 3374496 } seq expand size = 6516230 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4435 value = 3.5768 dict = 1.00 all = 3.6793 } Zip/UnZip{ index = 0.0958 value = 0.2796 dict = 1.00 all = 0.2718 } ---------------------------- total value len = 0.344163 GB avg = 0.124 KB (by entry num) total key len = 0.077988 GB avg = 0.028 KB total ukey len = 0.055705 GB avg = 0.020 KB total ukey num = 0.002785268 Billion total entry num = 0.002785268 Billion write speed all = 0.005412327 MB/sec (with seq num) write speed all = 0.005126651 MB/sec (without seq num) 2024/11/23-06:32:21.068298 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248720: 930890 keys, 36478982 bytes, temperature: kWarm 2024/11/23-06:32:21.068350 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314741068319, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248720, "file_size": 36478982, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3837398055, "table_properties": {"data_size": 30237648, "index_size": 1782720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26064920, "raw_average_key_size": 28, "raw_value_size": 108152876, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930890, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314737, "oldest_key_time": 0, "file_creation_time": 1732314737, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248720, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:21.104154 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248721.sst) /dev/shm/ToplingTemp/Topling-435562-bjnZdl 2024/11/23-06:32:21.895789 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248721.sst): first pass time = 0.79's, 169.553'MB/sec 2024/11/23-06:32:24.102116 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248721.sst): old prealloc_size = 147639500, real_size = 32428816 2024/11/23-06:32:24.102687 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248721.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932184 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.98 avg-zval = 32.64 2024/11/23-06:32:24.103495 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248721.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:25.407935 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248721.sst): second pass time = 2.20's, 49.042'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.31's, 28.073'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3230.348'MB/sec (index lex order gen) rebuild zvType time = 1.10's, 0.212'MB/sec write SST data time = 0.21's, 178.635'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30105.487'MB/sec zip my value time = 2.20's, unzip length = 0.108'GB zip my value throughput = 49.042'MB/sec zip pipeline throughput = 98.065'MB/sec entries = 932184 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.98 avg-zval = 32.64 usrkeys = 932184 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.98 avg-zval = 32.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932184 bytes = 3379184 } seq expand size = 6525288 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5139 value = 3.5539 dict = 1.00 all = 3.6613 } Zip/UnZip{ index = 0.0951 value = 0.2814 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 0.458805 GB avg = 0.123 KB (by entry num) total key len = 0.104089 GB avg = 0.028 KB total ukey len = 0.074349 GB avg = 0.020 KB total ukey num = 0.003717452 Billion total entry num = 0.003717452 Billion write speed all = 0.007216368 MB/sec (with seq num) write speed all = 0.006835102 MB/sec (without seq num) 2024/11/23-06:32:25.447121 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248721: 932184 keys, 36658408 bytes, temperature: kWarm 2024/11/23-06:32:25.447183 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314745447151, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248721, "file_size": 36658408, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3837397910, "table_properties": {"data_size": 30422336, "index_size": 1773248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26101152, "raw_average_key_size": 28, "raw_value_size": 108116604, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932184, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314741, "oldest_key_time": 0, "file_creation_time": 1732314741, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248721, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:25.481596 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248722.sst) /dev/shm/ToplingTemp/Topling-435562-x2wlgl 2024/11/23-06:32:26.327944 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248722.sst): first pass time = 0.85's, 158.592'MB/sec 2024/11/23-06:32:28.452611 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248722.sst): old prealloc_size = 147639500, real_size = 31320304 2024/11/23-06:32:28.453166 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248722.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921006 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.73 avg-zval = 31.83 2024/11/23-06:32:28.453963 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248722.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:29.482477 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248722.sst): second pass time = 2.12's, 51.052'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 34.487'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3330.249'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.257'MB/sec write SST data time = 0.13's, 266.237'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 28577.199'MB/sec zip my value time = 2.12's, unzip length = 0.108'GB zip my value throughput = 51.052'MB/sec zip pipeline throughput = 70.055'MB/sec entries = 921006 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.73 avg-zval = 31.83 usrkeys = 921006 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.73 avg-zval = 31.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921006 bytes = 3338656 } seq expand size = 6447042 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3758 value = 3.6988 dict = 1.00 all = 3.7790 } Zip/UnZip{ index = 0.0964 value = 0.2704 dict = 1.00 all = 0.2646 } ---------------------------- total value len = 0.573681 GB avg = 0.124 KB (by entry num) total key len = 0.129877 GB avg = 0.028 KB total ukey len = 0.092769 GB avg = 0.020 KB total ukey num = 0.004638458 Billion total entry num = 0.004638458 Billion write speed all = 0.009019238 MB/sec (with seq num) write speed all = 0.008543538 MB/sec (without seq num) 2024/11/23-06:32:29.515094 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248722: 921006 keys, 35516788 bytes, temperature: kWarm 2024/11/23-06:32:29.515149 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314749515120, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248722, "file_size": 35516788, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3439272420, "largest_seqno": 3837396742, "table_properties": {"data_size": 29314592, "index_size": 1775296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25788168, "raw_average_key_size": 28, "raw_value_size": 108429564, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921006, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314745, "oldest_key_time": 0, "file_creation_time": 1732314745, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248722, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:29.551567 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248723.sst) /dev/shm/ToplingTemp/Topling-435562-hYbrTj 2024/11/23-06:32:30.463328 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248723.sst): first pass time = 0.91's, 147.213'MB/sec 2024/11/23-06:32:32.157601 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248723.sst): old prealloc_size = 147639500, real_size = 33241760 2024/11/23-06:32:32.158314 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248723.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929889 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.34 avg-zval = 33.58 2024/11/23-06:32:32.159158 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248723.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:33.276311 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248723.sst): second pass time = 1.69's, 63.879'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 33.490'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2631.276'MB/sec (index lex order gen) rebuild zvType time = 0.96's, 0.242'MB/sec write SST data time = 0.16's, 236.149'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32134.302'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 63.879'MB/sec zip pipeline throughput = 68.693'MB/sec entries = 929889 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.34 avg-zval = 33.58 usrkeys = 929889 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.34 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929889 bytes = 3370864 } seq expand size = 6509223 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4469 value = 3.4641 dict = 1.00 all = 3.5825 } Zip/UnZip{ index = 0.0957 value = 0.2887 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 0.688371 GB avg = 0.124 KB (by entry num) total key len = 0.155914 GB avg = 0.028 KB total ukey len = 0.111367 GB avg = 0.020 KB total ukey num = 0.005568347 Billion total entry num = 0.005568347 Billion write speed all = 0.010822756 MB/sec (with seq num) write speed all = 0.010251718 MB/sec (without seq num) 2024/11/23-06:32:33.312199 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248723: 929889 keys, 37465102 bytes, temperature: kWarm 2024/11/23-06:32:33.312263 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314753312231, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248723, "file_size": 37465102, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3837397619, "table_properties": {"data_size": 31228880, "index_size": 1780224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26036892, "raw_average_key_size": 28, "raw_value_size": 108180874, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929889, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314749, "oldest_key_time": 0, "file_creation_time": 1732314749, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248723, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:33.355005 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248724.sst) /dev/shm/ToplingTemp/Topling-435562-XZtUMm 2024/11/23-06:32:34.408426 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248724.sst): first pass time = 1.05's, 127.416'MB/sec 2024/11/23-06:32:36.142406 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248724.sst): old prealloc_size = 147639500, real_size = 30702832 2024/11/23-06:32:36.143002 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248724.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928118 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0287 GB avg-val = 116.61 avg-zval = 30.93 2024/11/23-06:32:36.143794 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248724.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:37.005913 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248724.sst): second pass time = 1.73's, 62.439'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.174'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3071.903'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 265.159'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36537.967'MB/sec zip my value time = 1.73's, unzip length = 0.108'GB zip my value throughput = 62.439'MB/sec zip pipeline throughput = 74.352'MB/sec entries = 928118 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.61 avg-zval = 30.93 usrkeys = 928118 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.61 avg-zval = 30.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928118 bytes = 3132416 } seq expand size = 6496826 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.5128 value = 3.7704 dict = 1.00 all = 3.8692 } Zip/UnZip{ index = 0.0951 value = 0.2652 dict = 1.00 all = 0.2585 } ---------------------------- total value len = 0.803099 GB avg = 0.124 KB (by entry num) total key len = 0.181901 GB avg = 0.028 KB total ukey len = 0.129929 GB avg = 0.020 KB total ukey num = 0.006496465 Billion total entry num = 0.006496465 Billion write speed all = 0.012625950 MB/sec (with seq num) write speed all = 0.011959765 MB/sec (without seq num) 2024/11/23-06:32:37.039384 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248724: 928118 keys, 34688874 bytes, temperature: kWarm 2024/11/23-06:32:37.039466 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314757039421, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248724, "file_size": 34688874, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3717770124, "largest_seqno": 3837397706, "table_properties": {"data_size": 28704928, "index_size": 1765696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25987304, "raw_average_key_size": 28, "raw_value_size": 108230428, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928118, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314753, "oldest_key_time": 0, "file_creation_time": 1732314753, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248724, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:37.113761 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248725.sst) /dev/shm/ToplingTemp/Topling-435562-7Wmbbm 2024/11/23-06:32:38.054721 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248725.sst): first pass time = 0.94's, 142.648'MB/sec 2024/11/23-06:32:39.498397 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248725.sst): old prealloc_size = 147639500, real_size = 32653184 2024/11/23-06:32:39.499000 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248725.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929525 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.39 avg-zval = 32.95 2024/11/23-06:32:39.499745 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248725.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:40.380648 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248725.sst): second pass time = 1.44's, 74.974'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.434'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3063.706'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.13's, 272.842'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36601.397'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 74.974'MB/sec zip pipeline throughput = 74.974'MB/sec entries = 929525 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.39 avg-zval = 32.95 usrkeys = 929525 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.22 avg-zval = 32.95 TagRS{ kinds = RS_Key0_TagN bytes = 145320 } TagArray{ size = 773069 bytes = 2899024 } seq expand size = 5411483 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.3760 value = 3.5323 dict = 1.00 all = 3.6717 } Zip/UnZip{ index = 0.0964 value = 0.2831 dict = 1.00 all = 0.2724 } ---------------------------- total value len = 0.916701 GB avg = 0.123 KB (by entry num) total key len = 0.207928 GB avg = 0.028 KB total ukey len = 0.148520 GB avg = 0.020 KB total ukey num = 0.007425990 Billion total entry num = 0.007425990 Billion write speed all = 0.014415127 MB/sec (with seq num) write speed all = 0.013653655 MB/sec (without seq num) 2024/11/23-06:32:40.418411 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248725: 929525 keys, 36554500 bytes, temperature: kWarm 2024/11/23-06:32:40.418480 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314760418442, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248725, "file_size": 36554500, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3837397995, "table_properties": {"data_size": 30628912, "index_size": 1791680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26026700, "raw_average_key_size": 28, "raw_value_size": 108191130, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929525, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314757, "oldest_key_time": 0, "file_creation_time": 1732314757, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248725, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:40.452922 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248726.sst) /dev/shm/ToplingTemp/Topling-435562-4RaeCl 2024/11/23-06:32:41.282819 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248726.sst): first pass time = 0.83's, 161.737'MB/sec 2024/11/23-06:32:43.017441 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248726.sst): old prealloc_size = 147639500, real_size = 36468992 2024/11/23-06:32:43.017992 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248726.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927082 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.20 2024/11/23-06:32:43.018036 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248726.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:43.659383 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248726.sst): second pass time = 1.73's, 62.436'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.149'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3384.884'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.449'MB/sec write SST data time = 0.13's, 298.089'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 24892.094'MB/sec zip my value time = 1.73's, unzip length = 0.108'GB zip my value throughput = 62.436'MB/sec zip pipeline throughput = 88.589'MB/sec entries = 927082 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.20 usrkeys = 927082 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.77 avg-zval = 37.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5820 value = 3.1393 dict = 1.00 all = 3.5957 } Zip/UnZip{ index = 0.0945 value = 0.3185 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 1.024961 GB avg = 0.123 KB (by entry num) total key len = 0.233886 GB avg = 0.028 KB total ukey len = 0.167061 GB avg = 0.020 KB total ukey num = 0.008353072 Billion total entry num = 0.008353072 Billion write speed all = 0.016134809 MB/sec (with seq num) write speed all = 0.015278309 MB/sec (without seq num) 2024/11/23-06:32:43.691614 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248726: 927082 keys, 37327238 bytes, temperature: kWarm 2024/11/23-06:32:43.691678 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314763691645, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248726, "file_size": 37327238, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34484848, "index_size": 1752192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958296, "raw_average_key_size": 28, "raw_value_size": 108259524, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927082, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314760, "oldest_key_time": 0, "file_creation_time": 1732314760, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248726, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:43.725802 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248727.sst) /dev/shm/ToplingTemp/Topling-435562-nKNqIl 2024/11/23-06:32:44.500913 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248727.sst): first pass time = 0.78's, 173.168'MB/sec 2024/11/23-06:32:46.143625 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248727.sst): old prealloc_size = 147639500, real_size = 36039168 2024/11/23-06:32:46.144165 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248727.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926127 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0340 GB avg-val = 116.92 avg-zval = 36.76 2024/11/23-06:32:46.144610 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248727.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:46.907983 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248727.sst): second pass time = 1.64's, 65.946'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 50.329'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3390.758'MB/sec (index lex order gen) rebuild zvType time = 0.63's, 0.365'MB/sec write SST data time = 0.13's, 297.025'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33016.252'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.946'MB/sec zip pipeline throughput = 89.648'MB/sec entries = 926127 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.92 avg-zval = 36.76 usrkeys = 926127 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.80 avg-zval = 36.76 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 380174 bytes = 1425664 } seq expand size = 2661218 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0340 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5119 value = 3.1806 dict = 1.00 all = 3.4890 } Zip/UnZip{ index = 0.0951 value = 0.3144 dict = 1.00 all = 0.2866 } ---------------------------- total value len = 1.135908 GB avg = 0.122 KB (by entry num) total key len = 0.259818 GB avg = 0.028 KB total ukey len = 0.185584 GB avg = 0.020 KB total ukey num = 0.009279199 Billion total entry num = 0.009279199 Billion write speed all = 0.017888460 MB/sec (with seq num) write speed all = 0.016937038 MB/sec (without seq num) 2024/11/23-06:32:46.943913 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248727: 926127 keys, 38468417 bytes, temperature: kWarm 2024/11/23-06:32:46.943977 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314766943946, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248727, "file_size": 38468417, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3837397269, "table_properties": {"data_size": 34045424, "index_size": 1762048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931556, "raw_average_key_size": 28, "raw_value_size": 108286246, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926127, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314763, "oldest_key_time": 0, "file_creation_time": 1732314763, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248727, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:46.978027 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248728.sst) /dev/shm/ToplingTemp/Topling-435562-bLH16k 2024/11/23-06:32:47.827247 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248728.sst): first pass time = 0.85's, 158.055'MB/sec 2024/11/23-06:32:49.261020 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248728.sst): old prealloc_size = 147639500, real_size = 31939744 2024/11/23-06:32:49.261661 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248728.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929614 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0299 GB avg-val = 116.38 avg-zval = 32.19 2024/11/23-06:32:49.263825 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248728.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:50.177676 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248728.sst): second pass time = 1.43's, 75.499'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 39.451'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2875.348'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.14's, 265.382'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34374.570'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.499'MB/sec zip pipeline throughput = 75.499'MB/sec entries = 929614 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.38 avg-zval = 32.19 usrkeys = 929614 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.38 avg-zval = 32.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929614 bytes = 3369872 } seq expand size = 6507298 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4430 value = 3.6151 dict = 1.00 all = 3.7115 } Zip/UnZip{ index = 0.0958 value = 0.2766 dict = 1.00 all = 0.2694 } ---------------------------- total value len = 1.250604 GB avg = 0.123 KB (by entry num) total key len = 0.285847 GB avg = 0.028 KB total ukey len = 0.204176 GB avg = 0.020 KB total ukey num = 0.010208813 Billion total entry num = 0.010208813 Billion write speed all = 0.019691252 MB/sec (with seq num) write speed all = 0.018644557 MB/sec (without seq num) 2024/11/23-06:32:50.213680 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248728: 929614 keys, 36162248 bytes, temperature: kWarm 2024/11/23-06:32:50.213746 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314770213712, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248728, "file_size": 36162248, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3509327862, "largest_seqno": 3837397765, "table_properties": {"data_size": 29926800, "index_size": 1780352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26029192, "raw_average_key_size": 28, "raw_value_size": 108188544, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929614, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314766, "oldest_key_time": 0, "file_creation_time": 1732314766, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248728, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:50.252314 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248729.sst) /dev/shm/ToplingTemp/Topling-435562-YqJszn 2024/11/23-06:32:51.122842 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248729.sst): first pass time = 0.87's, 154.187'MB/sec 2024/11/23-06:32:52.680618 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248729.sst): old prealloc_size = 147639500, real_size = 32307184 2024/11/23-06:32:52.681154 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248729.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921773 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0303 GB avg-val = 117.61 avg-zval = 32.90 2024/11/23-06:32:52.683127 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248729.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:53.575661 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248729.sst): second pass time = 1.56's, 69.624'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.661'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3426.316'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.302'MB/sec write SST data time = 0.13's, 278.581'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32177.585'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.624'MB/sec zip pipeline throughput = 110.235'MB/sec entries = 921773 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.61 avg-zval = 32.90 usrkeys = 921773 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.61 avg-zval = 32.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921773 bytes = 3226224 } seq expand size = 6452411 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5114 value = 3.5751 dict = 1.00 all = 3.6881 } Zip/UnZip{ index = 0.0951 value = 0.2797 dict = 1.00 all = 0.2711 } ---------------------------- total value len = 1.365465 GB avg = 0.123 KB (by entry num) total key len = 0.311656 GB avg = 0.028 KB total ukey len = 0.222612 GB avg = 0.020 KB total ukey num = 0.011130586 Billion total entry num = 0.011130586 Billion write speed all = 0.021493154 MB/sec (with seq num) write speed all = 0.020352002 MB/sec (without seq num) 2024/11/23-06:32:53.607850 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248729: 921773 keys, 36392364 bytes, temperature: kWarm 2024/11/23-06:32:53.607914 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314773607881, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248729, "file_size": 36392364, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3837398081, "table_properties": {"data_size": 30322720, "index_size": 1753856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25809644, "raw_average_key_size": 28, "raw_value_size": 108408194, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921773, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314770, "oldest_key_time": 0, "file_creation_time": 1732314770, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248729, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:53.645046 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248730.sst) /dev/shm/ToplingTemp/Topling-435562-TBuLYk 2024/11/23-06:32:54.519554 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248730.sst): first pass time = 0.87's, 153.485'MB/sec 2024/11/23-06:32:56.166003 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248730.sst): old prealloc_size = 147639500, real_size = 33091184 2024/11/23-06:32:56.166550 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248730.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928401 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.57 avg-zval = 33.49 2024/11/23-06:32:56.168522 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248730.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:32:57.128366 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248730.sst): second pass time = 1.65's, 65.780'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.529'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3351.851'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.281'MB/sec write SST data time = 0.13's, 275.382'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33252.756'MB/sec zip my value time = 1.65's, unzip length = 0.108'GB zip my value throughput = 65.780'MB/sec zip pipeline throughput = 131.581'MB/sec entries = 928401 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.57 avg-zval = 33.49 usrkeys = 928401 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.57 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928401 bytes = 3133376 } seq expand size = 6498807 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5133 value = 3.4806 dict = 1.00 all = 3.6199 } Zip/UnZip{ index = 0.0951 value = 0.2873 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 1.480186 GB avg = 0.123 KB (by entry num) total key len = 0.337652 GB avg = 0.028 KB total ukey len = 0.241180 GB avg = 0.020 KB total ukey num = 0.012058987 Billion total entry num = 0.012058987 Billion write speed all = 0.023295448 MB/sec (with seq num) write speed all = 0.022059168 MB/sec (without seq num) 2024/11/23-06:32:57.180366 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248730: 928401 keys, 37078202 bytes, temperature: kWarm 2024/11/23-06:32:57.180430 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314777180398, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248730, "file_size": 37078202, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3736072251, "largest_seqno": 3837397302, "table_properties": {"data_size": 31092768, "index_size": 1766144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995228, "raw_average_key_size": 28, "raw_value_size": 108222558, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928401, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314773, "oldest_key_time": 0, "file_creation_time": 1732314773, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248730, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:32:57.218188 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248731.sst) /dev/shm/ToplingTemp/Topling-435562-wfwEYn 2024/11/23-06:32:58.052737 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248731.sst): first pass time = 0.83's, 160.833'MB/sec 2024/11/23-06:32:59.646181 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248731.sst): old prealloc_size = 147639500, real_size = 33228848 2024/11/23-06:32:59.646734 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248731.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928749 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.51 avg-zval = 33.61 2024/11/23-06:32:59.649307 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248731.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:33:00.611313 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248731.sst): second pass time = 1.59's, 67.968'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.682'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3359.570'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.278'MB/sec write SST data time = 0.13's, 286.737'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30840.018'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 67.968'MB/sec zip pipeline throughput = 110.608'MB/sec entries = 928749 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.51 avg-zval = 33.61 usrkeys = 928749 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.51 avg-zval = 33.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928749 bytes = 3250640 } seq expand size = 6501243 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4442 value = 3.4664 dict = 1.00 all = 3.5952 } Zip/UnZip{ index = 0.0957 value = 0.2885 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 1.594900 GB avg = 0.123 KB (by entry num) total key len = 0.363657 GB avg = 0.028 KB total ukey len = 0.259755 GB avg = 0.020 KB total ukey num = 0.012987736 Billion total entry num = 0.012987736 Billion write speed all = 0.025097631 MB/sec (with seq num) write speed all = 0.023766196 MB/sec (without seq num) 2024/11/23-06:33:00.655452 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248731: 928749 keys, 37332532 bytes, temperature: kWarm 2024/11/23-06:33:00.655518 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314780655485, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248731, "file_size": 37332532, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3837397690, "table_properties": {"data_size": 31217952, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26004972, "raw_average_key_size": 28, "raw_value_size": 108212786, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928749, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314777, "oldest_key_time": 0, "file_creation_time": 1732314777, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248731, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:33:00.694487 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248732.sst) /dev/shm/ToplingTemp/Topling-435562-TSLe7l 2024/11/23-06:33:01.586077 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248732.sst): first pass time = 0.89's, 150.544'MB/sec 2024/11/23-06:33:03.132274 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248732.sst): old prealloc_size = 147639500, real_size = 32285136 2024/11/23-06:33:03.132857 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248732.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934495 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 115.63 avg-zval = 32.40 2024/11/23-06:33:03.135233 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248732.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:33:04.088345 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248732.sst): second pass time = 1.55's, 69.913'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.080'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3161.743'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.283'MB/sec write SST data time = 0.13's, 280.611'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32561.203'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.913'MB/sec zip pipeline throughput = 129.475'MB/sec entries = 934495 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.63 avg-zval = 32.40 usrkeys = 934495 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.63 avg-zval = 32.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934495 bytes = 3270752 } seq expand size = 6541465 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5134 value = 3.5692 dict = 1.00 all = 3.6867 } Zip/UnZip{ index = 0.0951 value = 0.2802 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 1.709493 GB avg = 0.123 KB (by entry num) total key len = 0.389822 GB avg = 0.028 KB total ukey len = 0.278445 GB avg = 0.020 KB total ukey num = 0.013922231 Billion total entry num = 0.013922231 Billion write speed all = 0.026900170 MB/sec (with seq num) write speed all = 0.025472999 MB/sec (without seq num) 2024/11/23-06:33:04.122209 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248732: 934495 keys, 36406278 bytes, temperature: kWarm 2024/11/23-06:33:04.122285 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314784122242, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248732, "file_size": 36406278, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3837397342, "table_properties": {"data_size": 30273600, "index_size": 1777728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26165860, "raw_average_key_size": 28, "raw_value_size": 108051874, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934495, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314780, "oldest_key_time": 0, "file_creation_time": 1732314780, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248732, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:33:04.172205 140024743433792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248733.sst) /dev/shm/ToplingTemp/Topling-435562-vblAel 2024/11/23-06:33:04.265874 140024743433792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248733.sst): first pass time = 0.09's, 235.823'MB/sec 2024/11/23-06:33:04.463734 140024743433792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248733.sst): old prealloc_size = 147639500, real_size = 6245120 2024/11/23-06:33:04.463886 140024743433792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248733.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 154686 prefix = 4 raw-key = 0.0025 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0177 GB zip-val = 0.0059 GB avg-val = 114.74 avg-zval = 38.21 2024/11/23-06:33:04.464991 140024743433792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248733.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.022 GB 2024/11/23-06:33:04.610753 140024743433792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00/248733.sst): second pass time = 0.20's, 89.954'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 47.314'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2340.629'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.316'MB/sec write SST data time = 0.02's, 285.692'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.186'MB, throughput = 6250.344'MB/sec zip my value time = 0.20's, unzip length = 0.018'GB zip my value throughput = 89.954'MB/sec zip pipeline throughput = 89.954'MB/sec entries = 154686 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.74 avg-zval = 38.21 usrkeys = 154686 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.74 avg-zval = 38.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 154686 bytes = 560752 } seq expand size = 1082802 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0031 GB value = 0.0177 GB dict = 0.19 MB all = 0.0221 GB } __ZipSize{ index = 0.0003 GB value = 0.0059 GB dict = 0.19 MB all = 0.0070 GB } UnZip/Zip{ index = 10.4699 value = 3.0027 dict = 1.00 all = 3.1745 } Zip/UnZip{ index = 0.0955 value = 0.3330 dict = 1.00 all = 0.3150 } ---------------------------- total value len = 1.728324 GB avg = 0.123 KB (by entry num) total key len = 0.394154 GB avg = 0.028 KB total ukey len = 0.281538 GB avg = 0.020 KB total ukey num = 0.014076917 Billion total entry num = 0.014076917 Billion write speed all = 0.027196783 MB/sec (with seq num) write speed all = 0.025753765 MB/sec (without seq num) 2024/11/23-06:33:04.626506 140024743433792 [compaction_job.cc:1948] [default] [JOB 21104] Generated table #248733: 154686 keys, 6955271 bytes, temperature: kWarm 2024/11/23-06:33:04.626565 140024743433792 EVENT_LOG_v1 {"time_micros": 1732314784626535, "cf_name": "default", "job": 21104, "event": "table_file_creation", "file_number": 248733, "file_size": 6955271, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3837397594, "table_properties": {"data_size": 5910768, "index_size": 295488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4331208, "raw_average_key_size": 28, "raw_value_size": 17748196, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 154686, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732314784, "oldest_key_time": 0, "file_creation_time": 1732314784, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 248733, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:33:04.640470 140024743433792 [compaction_job.cc:703] job-21104: subcompact[0], size: 0.555246 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21104/att-00 [248718,248719,248720,248721,248722,248723,248724,248725,248726,248727,248728,248729,248730,248731,248732,248733] 2024/11/23-06:33:04.747864 140024743433792 (Original Log Time 2024/11/23-06:32:11.044359) EVENT_LOG_v1 {"time_micros": 1732314731044319, "job": 21104, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [248677], "files_L3": [248605, 248606, 248607, 248609, 248610, 248611, 248612, 248634, 248635, 248636, 248637, 248638, 248639, 248640, 248702], "score": 0.991018, "input_data_size": 576458535, "oldest_snapshot_seqno": 3837555301} 2024/11/23-06:33:04.747868 140024743433792 (Original Log Time 2024/11/23-06:33:04.747025) [compaction_job.cc:2022] [default] [JOB 21104] Compacted 1@2 + 15@3 files to L3 => 555245814 bytes 2024/11/23-06:33:04.747869 140024743433792 (Original Log Time 2024/11/23-06:33:04.747797) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.8 rd, 10.4 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.2, 522.6 +0.0 blob) out(529.5 +0.0 blob), read-write-amplify(39.7) write-amplify(19.5) OK, records in: 14158022, records dropped: 81105 output_compression: Snappy 2024/11/23-06:33:04.747876 140024743433792 (Original Log Time 2024/11/23-06:33:04.747844) EVENT_LOG_v1 {"time_micros": 1732314784747822, "cf": "default", "job": 21104, "event": "compaction_finished", "compaction_time_micros": 53596119, "compaction_time_cpu_micros": 22871557, "output_level": 3, "num_output_files": 16, "total_output_size": 555245814, "num_input_records": 14158022, "num_output_records": 14076917, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 391493195, "file_range_sync_nanos": 0, "file_fsync_nanos": 15209, "file_prepare_write_nanos": 34869, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/23-06:33:04.869650 140024743433792 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732314730,"executesMs":53849,"compactionJobId":21104,"attempt":0,"compactionInputRawBytes":2046863080,"compactionInputZipBytes":576458535,"compactionOutputRawBytes":0,"compactionOutputZipBytes":555245814,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-7b9db9aaca9e104775aee9fc906140b4-8ee0a0f5c385f6a9-00"}}, response = 2024/11/23-06:33:04.869726 140024743433792 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21104/att-00: olev 3, work 53.850 s, result 7.089 ms, install 0.871 ms, input{raw 1.906 GiB zip 549.754 MiB}