2024/11/23-19:41:32.877800 139957399803456 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00: fork to child time = 0.007766 sec 2024/11/23-19:41:32.878506 139957399803456 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00 2024/11/23-19:41:32.885871 139957399803456 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00 2024/11/23-19:41:32.886758 139957399803456 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-19:41:32.887023 139957399803456 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-19:41:32.942939 139957399803456 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34432/att-00/MANIFEST-206347 2024/11/23-19:41:32.975232 139957399803456 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34432/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 277683, last_sequence is 4222369072, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 277527 2024/11/23-19:41:32.975244 139957399803456 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-19:41:32.975410 139957399803456 [version_set.cc:5721] Creating manifest 277683 2024/11/23-19:41:33.010324 139957399803456 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-19:41:33.034917 139957399803456 [compaction_job.cc:2437] [default] [JOB 34432] Compacting 1@2 + 23@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-19:41:33.034936 139957399803456 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 25777 Base level 2, inputs: [277662(31MB)], [277578(40MB) 277579(40MB) 277580(40MB) 277581(40MB) 277582(40MB) 277583(40MB) 277584(40MB) 277585(40MB) 277586(40MB) 277587(40MB) 277588(40MB) 277589(40MB) 277590(40MB) 277591(40MB) 277592(40MB) 277593(40MB) 277594(40MB) 277595(40MB) 277596(40MB) 277597(40MB) 277598(40MB) 277599(40MB) 277680(40MB)] 2024/11/23-19:41:33.042251 139957399803456 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:41:33.042282 139957399803456 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:41:33.042302 139957399803456 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:41:33.042308 139957399803456 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:41:33.042319 139957399803456 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:41:33.945838 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277684.sst) /dev/shm/ToplingTemp/Topling-169084-WkZTix 2024/11/23-19:41:34.139075 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277684.sst): first pass time = 0.19's, 694.792'MB/sec 2024/11/23-19:41:34.870505 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277684.sst): old prealloc_size = 147639500, real_size = 40583136 2024/11/23-19:41:34.870631 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277684.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.50 2024/11/23-19:41:34.870973 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277684.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:35.036622 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277684.sst): second pass time = 0.73's, 180.162'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 255.357'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2171.762'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.05's, 907.294'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46581.034'MB/sec zip my value time = 0.73's, unzip length = 0.132'GB zip my value throughput = 180.162'MB/sec zip pipeline throughput = 180.162'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.99 avg-zval = 311.50 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.99 avg-zval = 311.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6818 value = 3.2648 dict = 1.00 all = 3.1644 } Zip/UnZip{ index = 0.1497 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 8.981156 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679866 Billion total entry num = 0.052487595 Billion write speed all = 0.268793368 MB/sec (with seq num) write speed all = 0.257966390 MB/sec (without seq num) 2024/11/23-19:41:35.090865 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277684: 129431 keys, 42415754 bytes, temperature: kWarm 2024/11/23-19:41:35.090965 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362095090903, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277684, "file_size": 42415754, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4222233107, "table_properties": {"data_size": 40318160, "index_size": 232448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629836, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362093, "oldest_key_time": 0, "file_creation_time": 1732362093, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277684, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:35.127551 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277685.sst) /dev/shm/ToplingTemp/Topling-169084-xIv2Ox 2024/11/23-19:41:35.811679 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277685.sst): first pass time = 0.68's, 196.199'MB/sec 2024/11/23-19:41:36.647613 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277685.sst): old prealloc_size = 147639500, real_size = 40872416 2024/11/23-19:41:36.647740 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277685.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.96 avg-zval = 313.79 2024/11/23-19:41:36.647897 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277685.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:36.815100 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277685.sst): second pass time = 0.84's, 157.601'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 254.998'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2220.327'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.05's, 900.090'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42969.099'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 157.601'MB/sec zip pipeline throughput = 157.601'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1016.96 avg-zval = 313.79 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1023.96 avg-zval = 313.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129434 bytes = 485392 } seq expand size = 906038 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9221 value = 3.2409 dict = 1.00 all = 3.1429 } Zip/UnZip{ index = 0.1445 value = 0.3086 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 9.113691 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809300 Billion total entry num = 0.052617029 Billion write speed all = 0.272265014 MB/sec (with seq num) write speed all = 0.261411834 MB/sec (without seq num) 2024/11/23-19:41:36.868323 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277685: 129434 keys, 42705186 bytes, temperature: kWarm 2024/11/23-19:41:36.868393 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362096868357, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277685, "file_size": 42705186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4222247195, "table_properties": {"data_size": 40615504, "index_size": 224384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629719, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362095, "oldest_key_time": 0, "file_creation_time": 1732362095, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277685, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:36.906019 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277686.sst) /dev/shm/ToplingTemp/Topling-169084-IMzLWy 2024/11/23-19:41:37.675878 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277686.sst): first pass time = 0.77's, 174.351'MB/sec 2024/11/23-19:41:38.444534 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277686.sst): old prealloc_size = 147639500, real_size = 40587680 2024/11/23-19:41:38.444672 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277686.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.97 avg-zval = 311.53 2024/11/23-19:41:38.444835 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277686.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:38.604650 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277686.sst): second pass time = 0.77's, 171.612'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.976'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2381.688'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 984.181'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 37315.915'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 171.612'MB/sec zip pipeline throughput = 171.612'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.97 avg-zval = 311.53 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.97 avg-zval = 311.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6819 value = 3.2644 dict = 1.00 all = 3.1640 } Zip/UnZip{ index = 0.1497 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.246227 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938733 Billion total entry num = 0.052746462 Billion write speed all = 0.275736266 MB/sec (with seq num) write speed all = 0.264856890 MB/sec (without seq num) 2024/11/23-19:41:38.655792 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277686: 129433 keys, 42420657 bytes, temperature: kWarm 2024/11/23-19:41:38.655863 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362098655826, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277686, "file_size": 42420657, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874654, "largest_seqno": 4222241926, "table_properties": {"data_size": 40322704, "index_size": 232448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629859, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362096, "oldest_key_time": 0, "file_creation_time": 1732362096, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277686, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:38.691915 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277687.sst) /dev/shm/ToplingTemp/Topling-169084-hENb1x 2024/11/23-19:41:39.259314 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277687.sst): first pass time = 0.57's, 236.566'MB/sec 2024/11/23-19:41:40.076546 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277687.sst): old prealloc_size = 147639500, real_size = 40873408 2024/11/23-19:41:40.076660 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277687.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.96 avg-zval = 313.80 2024/11/23-19:41:40.076813 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277687.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:40.238084 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277687.sst): second pass time = 0.82's, 161.234'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.417'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2380.679'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 979.010'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43503.062'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 161.234'MB/sec zip pipeline throughput = 161.234'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1016.96 avg-zval = 313.80 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1023.96 avg-zval = 313.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129435 bytes = 485392 } seq expand size = 906045 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9202 value = 3.2408 dict = 1.00 all = 3.1428 } Zip/UnZip{ index = 0.1445 value = 0.3086 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 9.378763 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068168 Billion total entry num = 0.052875897 Billion write speed all = 0.279208320 MB/sec (with seq num) write speed all = 0.268302707 MB/sec (without seq num) 2024/11/23-19:41:40.289393 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277687: 129435 keys, 42706194 bytes, temperature: kWarm 2024/11/23-19:41:40.289462 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362100289428, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277687, "file_size": 42706194, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4222231609, "table_properties": {"data_size": 40616432, "index_size": 224448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629772, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362098, "oldest_key_time": 0, "file_creation_time": 1732362098, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277687, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:40.331915 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277688.sst) /dev/shm/ToplingTemp/Topling-169084-KUe5gx 2024/11/23-19:41:41.103789 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277688.sst): first pass time = 0.77's, 173.896'MB/sec 2024/11/23-19:41:41.956601 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277688.sst): old prealloc_size = 147639500, real_size = 40583616 2024/11/23-19:41:41.956725 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277688.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.97 avg-zval = 311.50 2024/11/23-19:41:41.956886 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277688.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:42.121337 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277688.sst): second pass time = 0.85's, 154.612'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 257.515'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2218.566'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.05's, 931.520'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36084.324'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 154.612'MB/sec zip pipeline throughput = 154.612'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.97 avg-zval = 311.50 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.97 avg-zval = 311.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6819 value = 3.2647 dict = 1.00 all = 3.1643 } Zip/UnZip{ index = 0.1497 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 9.511299 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197601 Billion total entry num = 0.053005330 Billion write speed all = 0.282678253 MB/sec (with seq num) write speed all = 0.271746475 MB/sec (without seq num) 2024/11/23-19:41:42.174340 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277688: 129433 keys, 42416315 bytes, temperature: kWarm 2024/11/23-19:41:42.174413 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362102174375, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277688, "file_size": 42416315, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237869923, "largest_seqno": 4222236675, "table_properties": {"data_size": 40318640, "index_size": 232448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629526, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362100, "oldest_key_time": 0, "file_creation_time": 1732362100, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277688, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:42.216900 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277689.sst) /dev/shm/ToplingTemp/Topling-169084-vQYo5v 2024/11/23-19:41:42.897693 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277689.sst): first pass time = 0.68's, 197.161'MB/sec 2024/11/23-19:41:43.705369 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277689.sst): old prealloc_size = 147639500, real_size = 40588848 2024/11/23-19:41:43.705497 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277689.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.55 2024/11/23-19:41:43.705663 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277689.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:43.868563 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277689.sst): second pass time = 0.81's, 163.235'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.985'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2085.478'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.05's, 920.528'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47699.432'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 163.235'MB/sec zip pipeline throughput = 163.235'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.00 avg-zval = 311.55 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.00 avg-zval = 311.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6799 value = 3.2643 dict = 1.00 all = 3.1639 } Zip/UnZip{ index = 0.1497 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.643834 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327030 Billion total entry num = 0.053134759 Billion write speed all = 0.286148847 MB/sec (with seq num) write speed all = 0.275190870 MB/sec (without seq num) 2024/11/23-19:41:43.920998 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277689: 129429 keys, 42421779 bytes, temperature: kWarm 2024/11/23-19:41:43.921069 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362103921032, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277689, "file_size": 42421779, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4222241378, "table_properties": {"data_size": 40323808, "index_size": 232512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629396, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362102, "oldest_key_time": 0, "file_creation_time": 1732362102, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277689, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:43.960702 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277690.sst) /dev/shm/ToplingTemp/Topling-169084-1NZ7dv 2024/11/23-19:41:44.568037 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277690.sst): first pass time = 0.61's, 221.012'MB/sec 2024/11/23-19:41:45.506331 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277690.sst): old prealloc_size = 147639500, real_size = 40859168 2024/11/23-19:41:45.506455 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277690.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1017.00 avg-zval = 313.68 2024/11/23-19:41:45.506615 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277690.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:45.669561 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277690.sst): second pass time = 0.94's, 140.414'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.577'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2278.736'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 955.814'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46867.292'MB/sec zip my value time = 0.94's, unzip length = 0.132'GB zip my value throughput = 140.414'MB/sec zip pipeline throughput = 140.414'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1017.00 avg-zval = 313.68 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1024.00 avg-zval = 313.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8226 value = 3.2422 dict = 1.00 all = 3.1439 } Zip/UnZip{ index = 0.1466 value = 0.3084 dict = 1.00 all = 0.3181 } ---------------------------- total value len = 9.776370 GB avg = 0.184 KB (by entry num) total key len = 1.458942 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456459 Billion total entry num = 0.053264188 Billion write speed all = 0.289618736 MB/sec (with seq num) write speed all = 0.278634576 MB/sec (without seq num) 2024/11/23-19:41:45.722625 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277690: 129429 keys, 42691958 bytes, temperature: kWarm 2024/11/23-19:41:45.722700 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362105722659, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277690, "file_size": 42691958, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237885133, "largest_seqno": 4222244417, "table_properties": {"data_size": 40598992, "index_size": 227648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629746, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362103, "oldest_key_time": 0, "file_creation_time": 1732362103, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277690, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:45.759522 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277691.sst) /dev/shm/ToplingTemp/Topling-169084-PcBHJw 2024/11/23-19:41:46.495023 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277691.sst): first pass time = 0.74's, 182.494'MB/sec 2024/11/23-19:41:47.354372 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277691.sst): old prealloc_size = 147639500, real_size = 40992384 2024/11/23-19:41:47.354490 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277691.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.01 avg-zval = 314.74 2024/11/23-19:41:47.354645 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277691.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:47.516021 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277691.sst): second pass time = 0.86's, 153.395'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.000'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2211.170'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 985.005'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 54325.764'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 153.395'MB/sec zip pipeline throughput = 153.395'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.01 avg-zval = 314.74 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.01 avg-zval = 314.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129428 bytes = 485376 } seq expand size = 905996 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9396 value = 3.2313 dict = 1.00 all = 3.1341 } Zip/UnZip{ index = 0.1441 value = 0.3095 dict = 1.00 all = 0.3191 } ---------------------------- total value len = 9.908905 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585887 Billion total entry num = 0.053393616 Billion write speed all = 0.293087954 MB/sec (with seq num) write speed all = 0.282077628 MB/sec (without seq num) 2024/11/23-19:41:47.567435 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277691: 129428 keys, 42825140 bytes, temperature: kWarm 2024/11/23-19:41:47.567502 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362107567467, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277691, "file_size": 42825140, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237880987, "largest_seqno": 4222245141, "table_properties": {"data_size": 40736048, "index_size": 223808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629563, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362105, "oldest_key_time": 0, "file_creation_time": 1732362105, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277691, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:47.611482 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277692.sst) /dev/shm/ToplingTemp/Topling-169084-jTEUzy 2024/11/23-19:41:48.345229 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277692.sst): first pass time = 0.73's, 182.931'MB/sec 2024/11/23-19:41:49.164880 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277692.sst): old prealloc_size = 147639500, real_size = 41028448 2024/11/23-19:41:49.165037 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277692.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.99 avg-zval = 315.04 2024/11/23-19:41:49.165212 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277692.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:49.325184 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277692.sst): second pass time = 0.82's, 160.885'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.416'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1637.994'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.04's, 981.916'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39738.660'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.885'MB/sec zip pipeline throughput = 160.885'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.99 avg-zval = 315.04 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.99 avg-zval = 315.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 7.0342 value = 3.2282 dict = 1.00 all = 3.1315 } Zip/UnZip{ index = 0.1422 value = 0.3098 dict = 1.00 all = 0.3193 } ---------------------------- total value len = 10.041441 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715317 Billion total entry num = 0.053523046 Billion write speed all = 0.296557129 MB/sec (with seq num) write speed all = 0.285520628 MB/sec (without seq num) 2024/11/23-19:41:49.378955 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277692: 129430 keys, 42861296 bytes, temperature: kWarm 2024/11/23-19:41:49.379040 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362109379003, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277692, "file_size": 42861296, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4222247537, "table_properties": {"data_size": 40775120, "index_size": 220800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629613, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362107, "oldest_key_time": 0, "file_creation_time": 1732362107, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277692, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:49.417271 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277693.sst) /dev/shm/ToplingTemp/Topling-169084-2Q9zCz 2024/11/23-19:41:50.014112 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277693.sst): first pass time = 0.60's, 224.895'MB/sec 2024/11/23-19:41:50.867892 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277693.sst): old prealloc_size = 147639500, real_size = 40754000 2024/11/23-19:41:50.868030 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277693.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.97 avg-zval = 312.84 2024/11/23-19:41:50.868199 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277693.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:51.034350 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277693.sst): second pass time = 0.85's, 154.289'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 255.874'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2000.297'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 892.246'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 31954.933'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 154.289'MB/sec zip pipeline throughput = 154.289'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1016.97 avg-zval = 312.84 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1023.97 avg-zval = 312.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7733 value = 3.2508 dict = 1.00 all = 3.1517 } Zip/UnZip{ index = 0.1476 value = 0.3076 dict = 1.00 all = 0.3173 } ---------------------------- total value len = 10.173977 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844750 Billion total entry num = 0.053652479 Billion write speed all = 0.300026768 MB/sec (with seq num) write speed all = 0.288964065 MB/sec (without seq num) 2024/11/23-19:41:51.086523 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277693: 129433 keys, 42586472 bytes, temperature: kWarm 2024/11/23-19:41:51.086591 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362111086558, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277693, "file_size": 42586472, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874127, "largest_seqno": 4222247542, "table_properties": {"data_size": 40492160, "index_size": 229312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131630070, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362109, "oldest_key_time": 0, "file_creation_time": 1732362109, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277693, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:51.127088 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277694.sst) /dev/shm/ToplingTemp/Topling-169084-kfAtsy 2024/11/23-19:41:51.834632 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277694.sst): first pass time = 0.71's, 189.708'MB/sec 2024/11/23-19:41:53.078256 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277694.sst): old prealloc_size = 147639500, real_size = 40586752 2024/11/23-19:41:53.078391 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277694.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.53 2024/11/23-19:41:53.078549 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277694.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:53.244917 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277694.sst): second pass time = 1.24's, 105.912'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 254.579'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2178.785'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.05's, 888.545'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 38873.459'MB/sec zip my value time = 1.24's, unzip length = 0.132'GB zip my value throughput = 105.912'MB/sec zip pipeline throughput = 172.061'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.00 avg-zval = 311.53 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.00 avg-zval = 311.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6818 value = 3.2645 dict = 1.00 all = 3.1641 } Zip/UnZip{ index = 0.1497 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 10.306513 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974180 Billion total entry num = 0.053781909 Billion write speed all = 0.303492178 MB/sec (with seq num) write speed all = 0.292403419 MB/sec (without seq num) 2024/11/23-19:41:53.296973 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277694: 129430 keys, 42419638 bytes, temperature: kWarm 2024/11/23-19:41:53.297055 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362113297021, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277694, "file_size": 42419638, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873877, "largest_seqno": 4222226829, "table_properties": {"data_size": 40321776, "index_size": 232448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131630086, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362111, "oldest_key_time": 0, "file_creation_time": 1732362111, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277694, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:53.334920 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277695.sst) /dev/shm/ToplingTemp/Topling-169084-aDBwlz 2024/11/23-19:41:53.950924 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277695.sst): first pass time = 0.62's, 217.903'MB/sec 2024/11/23-19:41:54.723792 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277695.sst): old prealloc_size = 147639500, real_size = 40585056 2024/11/23-19:41:54.723917 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277695.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.01 avg-zval = 311.53 2024/11/23-19:41:54.724095 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277695.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:54.891438 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277695.sst): second pass time = 0.77's, 170.451'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 253.056'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2199.650'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.05's, 895.684'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42528.858'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 170.451'MB/sec zip pipeline throughput = 170.451'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.01 avg-zval = 311.53 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.01 avg-zval = 311.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129428 bytes = 485376 } seq expand size = 905996 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6816 value = 3.2646 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1497 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 10.439049 GB avg = 0.194 KB (by entry num) total key len = 1.471885 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103608 Billion total entry num = 0.053911337 Billion write speed all = 0.306961638 MB/sec (with seq num) write speed all = 0.295846665 MB/sec (without seq num) 2024/11/23-19:41:54.944409 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277695: 129428 keys, 42417742 bytes, temperature: kWarm 2024/11/23-19:41:54.944474 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362114944441, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277695, "file_size": 42417742, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4222245652, "table_properties": {"data_size": 40320080, "index_size": 232448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629483, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362113, "oldest_key_time": 0, "file_creation_time": 1732362113, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277695, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:54.981396 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277696.sst) /dev/shm/ToplingTemp/Topling-169084-h2llav 2024/11/23-19:41:55.711094 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277696.sst): first pass time = 0.73's, 183.947'MB/sec 2024/11/23-19:41:56.926746 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277696.sst): old prealloc_size = 147639500, real_size = 41078048 2024/11/23-19:41:56.926893 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277696.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.00 avg-zval = 315.41 2024/11/23-19:41:56.927101 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277696.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:57.103043 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277696.sst): second pass time = 1.21's, 108.355'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 243.427'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1780.541'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.251'MB/sec write SST data time = 0.05's, 911.884'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48108.744'MB/sec zip my value time = 1.21's, unzip length = 0.132'GB zip my value throughput = 108.355'MB/sec zip pipeline throughput = 149.405'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 315.41 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1024.00 avg-zval = 315.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9897 value = 3.2244 dict = 1.00 all = 3.1279 } Zip/UnZip{ index = 0.1431 value = 0.3101 dict = 1.00 all = 0.3197 } ---------------------------- total value len = 10.571584 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233038 Billion total entry num = 0.054040767 Billion write speed all = 0.310426291 MB/sec (with seq num) write speed all = 0.299285268 MB/sec (without seq num) 2024/11/23-19:41:57.155266 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277696: 129430 keys, 42910441 bytes, temperature: kWarm 2024/11/23-19:41:57.155335 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362117155299, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277696, "file_size": 42910441, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237888398, "largest_seqno": 4222226907, "table_properties": {"data_size": 40823312, "index_size": 222208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629738, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362114, "oldest_key_time": 0, "file_creation_time": 1732362114, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277696, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:57.191828 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277697.sst) /dev/shm/ToplingTemp/Topling-169084-9Tjdqy 2024/11/23-19:41:57.822649 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277697.sst): first pass time = 0.63's, 212.782'MB/sec 2024/11/23-19:41:58.667595 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277697.sst): old prealloc_size = 147639500, real_size = 40595872 2024/11/23-19:41:58.667728 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277697.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.04 avg-zval = 311.62 2024/11/23-19:41:58.667888 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277697.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:41:58.836626 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277697.sst): second pass time = 0.84's, 155.920'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 251.070'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2175.188'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.265'MB/sec write SST data time = 0.05's, 908.446'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 40472.582'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 155.920'MB/sec zip pipeline throughput = 155.920'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.04 avg-zval = 311.62 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.04 avg-zval = 311.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129425 bytes = 485360 } seq expand size = 905975 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6833 value = 3.2637 dict = 1.00 all = 3.1634 } Zip/UnZip{ index = 0.1496 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.704120 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362463 Billion total entry num = 0.054170192 Billion write speed all = 0.313894415 MB/sec (with seq num) write speed all = 0.302727209 MB/sec (without seq num) 2024/11/23-19:41:58.896640 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277697: 129425 keys, 42428529 bytes, temperature: kWarm 2024/11/23-19:41:58.896714 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362118896672, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277697, "file_size": 42428529, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237871365, "largest_seqno": 4222241488, "table_properties": {"data_size": 40330960, "index_size": 232384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629874, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362117, "oldest_key_time": 0, "file_creation_time": 1732362117, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277697, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:41:58.934601 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277698.sst) /dev/shm/ToplingTemp/Topling-169084-3ewO8w 2024/11/23-19:41:59.610150 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277698.sst): first pass time = 0.68's, 198.693'MB/sec 2024/11/23-19:42:00.529517 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277698.sst): old prealloc_size = 147639500, real_size = 40512704 2024/11/23-19:42:00.529670 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277698.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.03 2024/11/23-19:42:00.529807 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277698.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:42:00.796280 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277698.sst): second pass time = 0.92's, 143.294'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.27's, 157.774'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1718.724'MB/sec (index lex order gen) rebuild zvType time = 0.20's, 0.162'MB/sec write SST data time = 0.07's, 633.810'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45446.229'MB/sec zip my value time = 0.92's, unzip length = 0.132'GB zip my value throughput = 143.294'MB/sec zip pipeline throughput = 143.294'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.00 avg-zval = 311.03 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1019.90 avg-zval = 311.03 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 53626 bytes = 201120 } seq expand size = 375382 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9357 value = 3.2698 dict = 1.00 all = 3.1895 } Zip/UnZip{ index = 0.1442 value = 0.3058 dict = 1.00 all = 0.3135 } ---------------------------- total value len = 10.836126 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021176 GB avg = 0.015 KB total ukey num = 0.067491893 Billion total entry num = 0.054299622 Billion write speed all = 0.317346712 MB/sec (with seq num) write speed all = 0.306153390 MB/sec (without seq num) 2024/11/23-19:42:00.858242 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277698: 129430 keys, 42081617 bytes, temperature: kWarm 2024/11/23-19:42:00.858311 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362120858277, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277698, "file_size": 42081617, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4222239857, "table_properties": {"data_size": 40256240, "index_size": 223936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629890, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362118, "oldest_key_time": 0, "file_creation_time": 1732362118, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277698, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:42:00.897205 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277699.sst) /dev/shm/ToplingTemp/Topling-169084-F0R86x 2024/11/23-19:42:01.429549 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277699.sst): first pass time = 0.53's, 252.149'MB/sec 2024/11/23-19:42:02.355397 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277699.sst): old prealloc_size = 147639500, real_size = 40984832 2024/11/23-19:42:02.355529 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277699.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1016.97 avg-zval = 314.69 2024/11/23-19:42:02.355677 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277699.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:42:02.524108 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277699.sst): second pass time = 0.92's, 142.396'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 253.363'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2079.887'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.266'MB/sec write SST data time = 0.05's, 913.526'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49146.704'MB/sec zip my value time = 0.92's, unzip length = 0.132'GB zip my value throughput = 142.396'MB/sec zip pipeline throughput = 156.695'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1016.97 avg-zval = 314.69 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1022.56 avg-zval = 314.69 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 103315 bytes = 387440 } seq expand size = 723205 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 7.0467 value = 3.2316 dict = 1.00 all = 3.1403 } Zip/UnZip{ index = 0.1419 value = 0.3094 dict = 1.00 all = 0.3184 } ---------------------------- total value len = 10.968478 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621326 Billion total entry num = 0.054429055 Billion write speed all = 0.320809528 MB/sec (with seq num) write speed all = 0.309590023 MB/sec (without seq num) 2024/11/23-19:42:02.584611 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277699: 129433 keys, 42739860 bytes, temperature: kWarm 2024/11/23-19:42:02.584684 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362122584647, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277699, "file_size": 42739860, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4222247748, "table_properties": {"data_size": 40731888, "index_size": 220416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629395, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362120, "oldest_key_time": 0, "file_creation_time": 1732362120, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277699, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:42:02.623165 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277700.sst) /dev/shm/ToplingTemp/Topling-169084-fHAOHv 2024/11/23-19:42:03.193568 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277700.sst): first pass time = 0.57's, 235.323'MB/sec 2024/11/23-19:42:04.002135 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277700.sst): old prealloc_size = 147639500, real_size = 40622384 2024/11/23-19:42:04.002271 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277700.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.01 avg-zval = 311.86 2024/11/23-19:42:04.002393 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277700.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:42:04.162308 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277700.sst): second pass time = 0.81's, 163.102'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.035'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2240.857'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.04's, 965.425'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45509.100'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 163.102'MB/sec zip pipeline throughput = 163.102'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1017.01 avg-zval = 311.86 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1021.26 avg-zval = 311.86 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 78547 bytes = 294560 } seq expand size = 549829 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0423 GB } UnZip/Zip{ index = 6.8767 value = 3.2611 dict = 1.00 all = 3.1742 } Zip/UnZip{ index = 0.1454 value = 0.3066 dict = 1.00 all = 0.3150 } ---------------------------- total value len = 11.100658 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750754 Billion total entry num = 0.054558483 Billion write speed all = 0.324268333 MB/sec (with seq num) write speed all = 0.313022624 MB/sec (without seq num) 2024/11/23-19:42:04.215559 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277700: 129428 keys, 42284475 bytes, temperature: kWarm 2024/11/23-19:42:04.215629 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362124215593, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277700, "file_size": 42284475, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4222246184, "table_properties": {"data_size": 40364000, "index_size": 225856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131630134, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362122, "oldest_key_time": 0, "file_creation_time": 1732362122, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277700, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:42:04.250666 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277701.sst) /dev/shm/ToplingTemp/Topling-169084-G6MDxy 2024/11/23-19:42:04.929259 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277701.sst): first pass time = 0.68's, 197.801'MB/sec 2024/11/23-19:42:05.793008 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277701.sst): old prealloc_size = 147639500, real_size = 41071296 2024/11/23-19:42:05.793123 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277701.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.04 avg-zval = 315.37 2024/11/23-19:42:05.793290 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277701.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:42:05.950969 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277701.sst): second pass time = 0.86's, 152.518'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 271.573'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2363.501'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.279'MB/sec write SST data time = 0.04's, 1028.842'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49788.344'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 152.518'MB/sec zip pipeline throughput = 152.518'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1017.04 avg-zval = 315.37 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.61 avg-zval = 315.37 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 121551 bytes = 455824 } seq expand size = 850857 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 7.0015 value = 3.2249 dict = 1.00 all = 3.1290 } Zip/UnZip{ index = 0.1428 value = 0.3101 dict = 1.00 all = 0.3196 } ---------------------------- total value len = 11.233139 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880179 Billion total entry num = 0.054687908 Billion write speed all = 0.327733324 MB/sec (with seq num) write speed all = 0.316461458 MB/sec (without seq num) 2024/11/23-19:42:06.003919 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277701: 129425 keys, 42894919 bytes, temperature: kWarm 2024/11/23-19:42:06.004001 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362126003953, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277701, "file_size": 42894919, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4222232378, "table_properties": {"data_size": 40816944, "index_size": 221824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629915, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362124, "oldest_key_time": 0, "file_creation_time": 1732362124, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277701, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:42:06.042263 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277702.sst) /dev/shm/ToplingTemp/Topling-169084-TFoQ0x 2024/11/23-19:42:06.703498 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277702.sst): first pass time = 0.66's, 202.993'MB/sec 2024/11/23-19:42:07.489696 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277702.sst): old prealloc_size = 147639500, real_size = 40543312 2024/11/23-19:42:07.489817 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277702.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.25 2024/11/23-19:42:07.489934 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277702.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:42:07.650032 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277702.sst): second pass time = 0.79's, 167.573'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.160'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2323.049'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.280'MB/sec write SST data time = 0.04's, 947.962'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45364.292'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 167.573'MB/sec zip pipeline throughput = 167.573'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1017.00 avg-zval = 311.25 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1020.92 avg-zval = 311.25 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 72464 bytes = 271760 } seq expand size = 507248 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 6.8553 value = 3.2675 dict = 1.00 all = 3.1818 } Zip/UnZip{ index = 0.1459 value = 0.3060 dict = 1.00 all = 0.3143 } ---------------------------- total value len = 11.365276 GB avg = 0.207 KB (by entry num) total key len = 1.490005 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009608 Billion total entry num = 0.054817337 Billion write speed all = 0.331189898 MB/sec (with seq num) write speed all = 0.319891849 MB/sec (without seq num) 2024/11/23-19:42:07.701671 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277702: 129429 keys, 42183129 bytes, temperature: kWarm 2024/11/23-19:42:07.701744 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362127701706, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277702, "file_size": 42183129, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4222236676, "table_properties": {"data_size": 40284224, "index_size": 226560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629413, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362126, "oldest_key_time": 0, "file_creation_time": 1732362126, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277702, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:42:07.736722 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277703.sst) /dev/shm/ToplingTemp/Topling-169084-98vI8v 2024/11/23-19:42:08.564056 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277703.sst): first pass time = 0.83's, 162.237'MB/sec 2024/11/23-19:42:09.366211 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277703.sst): old prealloc_size = 147639500, real_size = 40609472 2024/11/23-19:42:09.366339 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277703.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.02 avg-zval = 311.79 2024/11/23-19:42:09.366426 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277703.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:42:09.525230 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277703.sst): second pass time = 0.80's, 164.247'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.867'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2235.979'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 1010.952'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.377'MB, throughput = 50753.141'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.247'MB/sec zip pipeline throughput = 164.247'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.02 avg-zval = 311.79 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.98 avg-zval = 311.79 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 36307 bytes = 136160 } seq expand size = 254149 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9875 value = 3.2618 dict = 1.00 all = 3.1871 } Zip/UnZip{ index = 0.1431 value = 0.3066 dict = 1.00 all = 0.3138 } ---------------------------- total value len = 11.497159 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139035 Billion total entry num = 0.054946764 Billion write speed all = 0.334638124 MB/sec (with seq num) write speed all = 0.323313947 MB/sec (without seq num) 2024/11/23-19:42:09.577545 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277703: 129427 keys, 42112850 bytes, temperature: kWarm 2024/11/23-19:42:09.577621 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362129577579, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277703, "file_size": 42112850, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4222230768, "table_properties": {"data_size": 40354672, "index_size": 222272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629223, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362127, "oldest_key_time": 0, "file_creation_time": 1732362127, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277703, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:42:09.617155 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277704.sst) /dev/shm/ToplingTemp/Topling-169084-7aDU0v 2024/11/23-19:42:10.313422 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277704.sst): first pass time = 0.70's, 192.778'MB/sec 2024/11/23-19:42:11.099023 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277704.sst): old prealloc_size = 147639500, real_size = 40492464 2024/11/23-19:42:11.099170 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277704.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.88 2024/11/23-19:42:11.099267 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277704.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:42:11.257486 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277704.sst): second pass time = 0.78's, 167.873'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.103'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1939.512'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 1008.881'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36438.412'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 167.873'MB/sec zip pipeline throughput = 167.873'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 310.88 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1019.07 avg-zval = 310.88 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 38184 bytes = 143200 } seq expand size = 267288 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9755 value = 3.2713 dict = 1.00 all = 3.1954 } Zip/UnZip{ index = 0.1434 value = 0.3057 dict = 1.00 all = 0.3129 } ---------------------------- total value len = 11.629056 GB avg = 0.211 KB (by entry num) total key len = 1.495182 GB avg = 0.027 KB total ukey len = 1.030494 GB avg = 0.015 KB total ukey num = 0.068268464 Billion total entry num = 0.055076193 Billion write speed all = 0.338087612 MB/sec (with seq num) write speed all = 0.326737266 MB/sec (without seq num) 2024/11/23-19:42:11.309885 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277704: 129429 keys, 42003362 bytes, temperature: kWarm 2024/11/23-19:42:11.309955 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362131309919, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277704, "file_size": 42003362, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4222228267, "table_properties": {"data_size": 40237280, "index_size": 222656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629629, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362129, "oldest_key_time": 0, "file_creation_time": 1732362129, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277704, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:42:11.348204 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277705.sst) /dev/shm/ToplingTemp/Topling-169084-Li5k5y 2024/11/23-19:42:12.113507 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277705.sst): first pass time = 0.77's, 175.388'MB/sec 2024/11/23-19:42:12.962158 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277705.sst): old prealloc_size = 147639500, real_size = 40477968 2024/11/23-19:42:12.962274 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277705.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.77 2024/11/23-19:42:12.962356 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277705.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:42:13.126505 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277705.sst): second pass time = 0.85's, 155.219'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 255.463'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2324.986'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.04's, 966.776'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45856.397'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 155.219'MB/sec zip pipeline throughput = 155.219'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 310.77 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.92 avg-zval = 310.77 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 35695 bytes = 133872 } seq expand size = 249865 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9817 value = 3.2725 dict = 1.00 all = 3.1972 } Zip/UnZip{ index = 0.1432 value = 0.3056 dict = 1.00 all = 0.3128 } ---------------------------- total value len = 11.760936 GB avg = 0.213 KB (by entry num) total key len = 1.497770 GB avg = 0.027 KB total ukey len = 1.032048 GB avg = 0.015 KB total ukey num = 0.068397895 Billion total entry num = 0.055205624 Billion write speed all = 0.341535147 MB/sec (with seq num) write speed all = 0.330158676 MB/sec (without seq num) 2024/11/23-19:42:13.177870 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277705: 129431 keys, 41979749 bytes, temperature: kWarm 2024/11/23-19:42:13.177938 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362133177905, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277705, "file_size": 41979749, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4222246574, "table_properties": {"data_size": 40222976, "index_size": 222464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629883, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362131, "oldest_key_time": 0, "file_creation_time": 1732362131, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277705, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:42:13.219637 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277706.sst) /dev/shm/ToplingTemp/Topling-169084-xYJOiv 2024/11/23-19:42:13.927595 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277706.sst): first pass time = 0.71's, 189.595'MB/sec 2024/11/23-19:42:14.774191 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277706.sst): old prealloc_size = 147639500, real_size = 41127488 2024/11/23-19:42:14.774306 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277706.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0409 GB avg-val =1016.98 avg-zval = 315.79 2024/11/23-19:42:14.774472 139957399803456 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277706.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:42:14.942024 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277706.sst): second pass time = 0.85's, 155.612'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 256.044'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2346.893'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.266'MB/sec write SST data time = 0.05's, 934.378'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44838.025'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 155.612'MB/sec zip pipeline throughput = 155.612'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.98 avg-zval = 315.79 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.72 avg-zval = 315.79 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 124660 bytes = 467488 } seq expand size = 872620 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0409 GB dict = 1.38 MB all = 0.0430 GB } UnZip/Zip{ index = 7.0181 value = 3.2204 dict = 1.00 all = 3.1241 } Zip/UnZip{ index = 0.1425 value = 0.3105 dict = 1.00 all = 0.3201 } ---------------------------- total value len = 11.893438 GB avg = 0.215 KB (by entry num) total key len = 1.500359 GB avg = 0.027 KB total ukey len = 1.033601 GB avg = 0.015 KB total ukey num = 0.068527327 Billion total entry num = 0.055335056 Billion write speed all = 0.344998854 MB/sec (with seq num) write speed all = 0.333596243 MB/sec (without seq num) 2024/11/23-19:42:14.994250 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277706: 129432 keys, 42962441 bytes, temperature: kWarm 2024/11/23-19:42:14.994323 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362134994286, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277706, "file_size": 42962441, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4222247902, "table_properties": {"data_size": 40873648, "index_size": 221312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629327, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362133, "oldest_key_time": 0, "file_creation_time": 1732362133, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277706, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:42:15.033693 139957399803456 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277707.sst) /dev/shm/ToplingTemp/Topling-169084-kIvFaw 2024/11/23-19:42:15.041788 139957399803456 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277707.sst): first pass time = 0.01's, 334.512'MB/sec 2024/11/23-19:42:15.072771 139957399803456 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277707.sst): old prealloc_size = 147639500, real_size = 858272 2024/11/23-19:42:15.072807 139957399803456 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277707.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2600 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.28 raw-val = 0.0026 GB zip-val = 0.0009 GB avg-val =1017.09 avg-zval = 329.50 2024/11/23-19:42:15.075782 139957399803456 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00/277707.sst): second pass time = 0.03's, 86.605'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 300.380'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 32.693'MB/sec (index lex order gen) rebuild zvType time = 0.00's, 0.309'MB/sec write SST data time = 0.00's, 1040.247'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.033'MB, throughput = 1368.211'MB/sec zip my value time = 0.03's, unzip length = 0.003'GB zip my value throughput = 86.605'MB/sec zip pipeline throughput = 86.605'MB/sec entries = 2600 avg-key = 20.00 avg-zkey = 0.30 avg-val = 1017.09 avg-zval = 329.50 usrkeys = 2600 avg-key = 12.00 avg-zkey = 0.30 avg-val = 1024.09 avg-zval = 329.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2600 bytes = 7808 } seq expand size = 18200 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0000 GB value = 0.0026 GB dict = 0.03 MB all = 0.0027 GB } __ZipSize{ index = 0.0000 GB value = 0.0009 GB dict = 0.03 MB all = 0.0009 GB } UnZip/Zip{ index = 40.6250 value = 3.0867 dict = 1.00 all = 2.9992 } Zip/UnZip{ index = 0.0246 value = 0.3240 dict = 1.00 all = 0.3334 } ---------------------------- total value len = 11.896100 GB avg = 0.215 KB (by entry num) total key len = 1.500411 GB avg = 0.027 KB total ukey len = 1.033632 GB avg = 0.015 KB total ukey num = 0.068529927 Billion total entry num = 0.055337656 Billion write speed all = 0.345067588 MB/sec (with seq num) write speed all = 0.333664481 MB/sec (without seq num) 2024/11/23-19:42:15.082347 139957399803456 [compaction_job.cc:1948] [default] [JOB 34432] Generated table #277707: 2600 keys, 899044 bytes, temperature: kWarm 2024/11/23-19:42:15.082394 139957399803456 EVENT_LOG_v1 {"time_micros": 1732362135082368, "cf_name": "default", "job": 34432, "event": "table_file_creation", "file_number": 277707, "file_size": 899044, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4205759618, "largest_seqno": 4222247881, "table_properties": {"data_size": 856704, "index_size": 768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 52000, "raw_average_key_size": 20, "raw_value_size": 2644424, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 2600, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1732362135, "oldest_key_time": 0, "file_creation_time": 1732362135, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 277707, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:42:15.091595 139957399803456 [compaction_job.cc:703] job-34432: subcompact[0], size: 0.978369 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34432/att-00 [277684,277685,277686,277687,277688,277689,277690,277691,277692,277693,277694,277695,277696,277697,277698,277699,277700,277701,277702,277703,277704,277705,277706,277707] 2024/11/23-19:42:15.233817 139957399803456 (Original Log Time 2024/11/23-19:41:33.035009) EVENT_LOG_v1 {"time_micros": 1732362093034954, "job": 34432, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [277662], "files_L3": [277578, 277579, 277580, 277581, 277582, 277583, 277584, 277585, 277586, 277587, 277588, 277589, 277590, 277591, 277592, 277593, 277594, 277595, 277596, 277597, 277598, 277599, 277680], "score": 0.993118, "input_data_size": 1010140324, "oldest_snapshot_seqno": 4222369003} 2024/11/23-19:42:15.233825 139957399803456 (Original Log Time 2024/11/23-19:42:15.232217) [compaction_job.cc:2022] [default] [JOB 34432] Compacted 1@2 + 23@3 files to L3 => 978368547 bytes 2024/11/23-19:42:15.233826 139957399803456 (Original Log Time 2024/11/23-19:42:15.233739) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 24 0 0 0] max score 0.04, MB/sec: 24.0 rd, 23.3 wr, level 3, files in(1, 23) out(24 +0 blob) MB in(31.2, 932.2 +0.0 blob) out(933.0 +0.0 blob), read-write-amplify(60.8) write-amplify(29.9) OK, records in: 3008804, records dropped: 29312 output_compression: Snappy 2024/11/23-19:42:15.233829 139957399803456 (Original Log Time 2024/11/23-19:42:15.233790) EVENT_LOG_v1 {"time_micros": 1732362135233768, "cf": "default", "job": 34432, "event": "compaction_finished", "compaction_time_micros": 42056589, "compaction_time_cpu_micros": 8609227, "output_level": 3, "num_output_files": 24, "total_output_size": 978368547, "num_input_records": 3008804, "num_output_records": 2979492, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 689165635, "file_range_sync_nanos": 0, "file_fsync_nanos": 17465, "file_prepare_write_nanos": 53196, "lsm_state": [0, 0, 0, 24, 0, 0, 0]} 2024/11/23-19:42:15.356655 139957399803456 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732362092,"executesMs":42348,"compactionJobId":34432,"attempt":0,"compactionInputRawBytes":3120114292,"compactionInputZipBytes":1010140324,"compactionOutputRawBytes":0,"compactionOutputZipBytes":978368547,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-af596c1e7027961508ed4433cd0529f5-066c677ab5e0343b-00"}}, response = 2024/11/23-19:42:15.356744 139957399803456 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34432/att-00: olev 3, work 42.348 s, result 5.996 ms, install 1.629 ms, input{raw 2.906 GiB zip 963.345 MiB}