2024/11/23-14:37:33.828745 139957685155392 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00: fork to child time = 0.010979 sec 2024/11/23-14:37:33.829479 139957685155392 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00 2024/11/23-14:37:33.843633 139957685155392 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00 2024/11/23-14:37:33.844525 139957685155392 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:37:33.844791 139957685155392 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:37:33.900630 139957685155392 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28061/att-00/MANIFEST-206347 2024/11/23-14:37:33.933364 139957685155392 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28061/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 263820, last_sequence is 4030762458, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 263683 2024/11/23-14:37:33.933372 139957685155392 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:37:33.933506 139957685155392 [version_set.cc:5721] Creating manifest 263820 2024/11/23-14:37:33.968170 139957685155392 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:37:33.993939 139957685155392 [compaction_job.cc:2437] [default] [JOB 28061] Compacting 1@2 + 20@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-14:37:33.993960 139957685155392 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20964 Base level 2, inputs: [263816(31MB)], [263737(40MB) 263738(40MB) 263739(40MB) 263740(40MB) 263741(40MB) 263742(40MB) 263743(40MB) 263744(40MB) 263745(40MB) 263746(40MB) 263747(40MB) 263748(40MB) 263749(40MB) 263750(40MB) 263751(40MB) 263752(40MB) 263753(39MB) 263754(40MB) 263755(39MB) 263758(40MB)] 2024/11/23-14:37:34.001281 139957685155392 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:37:34.001311 139957685155392 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:37:34.001330 139957685155392 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:37:34.001335 139957685155392 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:37:34.001346 139957685155392 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:37:34.660536 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263821.sst) /dev/shm/ToplingTemp/Topling-76711-j7m8sI 2024/11/23-14:37:35.005425 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263821.sst): first pass time = 0.34's, 389.219'MB/sec 2024/11/23-14:37:35.850097 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263821.sst): old prealloc_size = 147639500, real_size = 40693024 2024/11/23-14:37:35.850218 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263821.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.98 avg-zval = 312.34 2024/11/23-14:37:35.850383 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263821.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:36.015314 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263821.sst): second pass time = 0.84's, 156.099'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 257.405'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2294.312'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.05's, 905.456'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 50129.497'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 156.099'MB/sec zip pipeline throughput = 156.099'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.98 avg-zval = 312.34 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.98 avg-zval = 312.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6489 value = 3.2560 dict = 1.00 all = 3.1561 } Zip/UnZip{ index = 0.1504 value = 0.3071 dict = 1.00 all = 0.3168 } ---------------------------- total value len = 8.981155 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679866 Billion total entry num = 0.052487595 Billion write speed all = 0.507431171 MB/sec (with seq num) write speed all = 0.486991878 MB/sec (without seq num) 2024/11/23-14:37:36.069121 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263821: 129431 keys, 42525885 bytes, temperature: kWarm 2024/11/23-14:37:36.069201 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343856069148, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263821, "file_size": 42525885, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4030463077, "table_properties": {"data_size": 40426896, "index_size": 233600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629333, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343854, "oldest_key_time": 0, "file_creation_time": 1732343854, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263821, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:36.107682 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263822.sst) /dev/shm/ToplingTemp/Topling-76711-zdQbiI 2024/11/23-14:37:36.832120 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263822.sst): first pass time = 0.72's, 185.282'MB/sec 2024/11/23-14:37:37.752173 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263822.sst): old prealloc_size = 147639500, real_size = 40815712 2024/11/23-14:37:37.752320 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263822.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.99 avg-zval = 313.33 2024/11/23-14:37:37.752503 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263822.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:37.928265 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263822.sst): second pass time = 0.92's, 143.203'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 242.216'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1919.946'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.249'MB/sec write SST data time = 0.05's, 926.292'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49284.314'MB/sec zip my value time = 0.92's, unzip length = 0.132'GB zip my value throughput = 143.203'MB/sec zip pipeline throughput = 143.203'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1016.99 avg-zval = 313.33 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1023.99 avg-zval = 313.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7940 value = 3.2457 dict = 1.00 all = 3.1471 } Zip/UnZip{ index = 0.1472 value = 0.3081 dict = 1.00 all = 0.3178 } ---------------------------- total value len = 9.113691 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809297 Billion total entry num = 0.052617026 Billion write speed all = 0.513960687 MB/sec (with seq num) write speed all = 0.493472900 MB/sec (without seq num) 2024/11/23-14:37:37.981704 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263822: 129431 keys, 42648551 bytes, temperature: kWarm 2024/11/23-14:37:37.981770 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343857981737, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263822, "file_size": 42648551, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4030470656, "table_properties": {"data_size": 40554576, "index_size": 228608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629612, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343856, "oldest_key_time": 0, "file_creation_time": 1732343856, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263822, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:38.022692 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263823.sst) /dev/shm/ToplingTemp/Topling-76711-6EaxkI 2024/11/23-14:37:38.995125 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263823.sst): first pass time = 0.97's, 138.030'MB/sec 2024/11/23-14:37:40.427852 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263823.sst): old prealloc_size = 147639500, real_size = 40708320 2024/11/23-14:37:40.427988 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263823.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.97 avg-zval = 312.46 2024/11/23-14:37:40.428159 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263823.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:40.599892 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263823.sst): second pass time = 1.43's, 91.914'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 247.303'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1941.336'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.259'MB/sec write SST data time = 0.05's, 903.175'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46871.319'MB/sec zip my value time = 1.43's, unzip length = 0.132'GB zip my value throughput = 91.914'MB/sec zip pipeline throughput = 113.754'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.97 avg-zval = 312.46 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.97 avg-zval = 312.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6764 value = 3.2547 dict = 1.00 all = 3.1550 } Zip/UnZip{ index = 0.1498 value = 0.3072 dict = 1.00 all = 0.3170 } ---------------------------- total value len = 9.246226 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938730 Billion total entry num = 0.052746459 Billion write speed all = 0.520469770 MB/sec (with seq num) write speed all = 0.499934255 MB/sec (without seq num) 2024/11/23-14:37:40.652808 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263823: 129433 keys, 42541369 bytes, temperature: kWarm 2024/11/23-14:37:40.652876 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343860652842, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263823, "file_size": 42541369, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237870807, "largest_seqno": 4030476393, "table_properties": {"data_size": 40443152, "index_size": 232640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629552, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343858, "oldest_key_time": 0, "file_creation_time": 1732343858, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263823, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:40.694828 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263824.sst) /dev/shm/ToplingTemp/Topling-76711-BJmviH 2024/11/23-14:37:41.345656 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263824.sst): first pass time = 0.65's, 206.242'MB/sec 2024/11/23-14:37:42.418134 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263824.sst): old prealloc_size = 147639500, real_size = 40796496 2024/11/23-14:37:42.418256 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263824.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.96 avg-zval = 313.16 2024/11/23-14:37:42.418421 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263824.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:42.582992 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263824.sst): second pass time = 1.07's, 122.918'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.629'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2265.274'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.05's, 908.757'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 54166.477'MB/sec zip my value time = 1.07's, unzip length = 0.132'GB zip my value throughput = 122.918'MB/sec zip pipeline throughput = 122.918'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1016.96 avg-zval = 313.16 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1023.96 avg-zval = 313.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129435 bytes = 485392 } seq expand size = 906045 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7677 value = 3.2474 dict = 1.00 all = 3.1485 } Zip/UnZip{ index = 0.1478 value = 0.3079 dict = 1.00 all = 0.3176 } ---------------------------- total value len = 9.378762 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068165 Billion total entry num = 0.052875894 Billion write speed all = 0.526994833 MB/sec (with seq num) write speed all = 0.506410912 MB/sec (without seq num) 2024/11/23-14:37:42.635337 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263824: 129435 keys, 42629204 bytes, temperature: kWarm 2024/11/23-14:37:42.635394 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343862635362, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263824, "file_size": 42629204, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4030478213, "table_properties": {"data_size": 40534464, "index_size": 229504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629845, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343860, "oldest_key_time": 0, "file_creation_time": 1732343860, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263824, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:42.678039 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263825.sst) /dev/shm/ToplingTemp/Topling-76711-98BScI 2024/11/23-14:37:43.508070 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263825.sst): first pass time = 0.83's, 161.708'MB/sec 2024/11/23-14:37:44.961302 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263825.sst): old prealloc_size = 147639500, real_size = 40593856 2024/11/23-14:37:44.961427 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263825.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.83 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.56 2024/11/23-14:37:44.961609 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263825.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:45.124419 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263825.sst): second pass time = 1.45's, 90.682'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.125'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2335.083'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 944.572'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43850.119'MB/sec zip my value time = 1.45's, unzip length = 0.132'GB zip my value throughput = 90.682'MB/sec zip pipeline throughput = 126.303'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.83 avg-val = 1017.00 avg-zval = 311.56 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.83 avg-val = 1024.00 avg-zval = 311.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5749 value = 3.2642 dict = 1.00 all = 3.1635 } Zip/UnZip{ index = 0.1521 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.511298 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197595 Billion total entry num = 0.053005324 Billion write speed all = 0.533504135 MB/sec (with seq num) write speed all = 0.512872380 MB/sec (without seq num) 2024/11/23-14:37:45.175840 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263825: 129430 keys, 42426629 bytes, temperature: kWarm 2024/11/23-14:37:45.175902 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343865175871, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263825, "file_size": 42426629, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4030474301, "table_properties": {"data_size": 40325104, "index_size": 236224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629815, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343862, "oldest_key_time": 0, "file_creation_time": 1732343862, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263825, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:45.223299 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263826.sst) /dev/shm/ToplingTemp/Topling-76711-mqnG0G 2024/11/23-14:37:45.874097 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263826.sst): first pass time = 0.65's, 206.249'MB/sec 2024/11/23-14:37:46.752857 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263826.sst): old prealloc_size = 147639500, real_size = 40964848 2024/11/23-14:37:46.753002 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263826.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.01 avg-zval = 314.50 2024/11/23-14:37:46.753169 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263826.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:46.919523 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263826.sst): second pass time = 0.88's, 149.909'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 256.831'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1728.834'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 889.181'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46055.331'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 149.909'MB/sec zip pipeline throughput = 149.909'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1017.01 avg-zval = 314.50 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1024.01 avg-zval = 314.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129428 bytes = 485376 } seq expand size = 905996 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.8379 value = 3.2337 dict = 1.00 all = 3.1361 } Zip/UnZip{ index = 0.1462 value = 0.3092 dict = 1.00 all = 0.3189 } ---------------------------- total value len = 9.643834 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327023 Billion total entry num = 0.053134752 Billion write speed all = 0.540031446 MB/sec (with seq num) write speed all = 0.519351119 MB/sec (without seq num) 2024/11/23-14:37:46.971882 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263826: 129428 keys, 42798037 bytes, temperature: kWarm 2024/11/23-14:37:46.971950 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343866971913, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263826, "file_size": 42798037, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237885133, "largest_seqno": 4030452237, "table_properties": {"data_size": 40705184, "index_size": 227136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131630093, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343865, "oldest_key_time": 0, "file_creation_time": 1732343865, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263826, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:47.018098 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263827.sst) /dev/shm/ToplingTemp/Topling-76711-TmVFqH 2024/11/23-14:37:47.887301 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263827.sst): first pass time = 0.87's, 154.423'MB/sec 2024/11/23-14:37:49.539274 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263827.sst): old prealloc_size = 147639500, real_size = 40984368 2024/11/23-14:37:49.539419 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263827.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.02 avg-zval = 314.68 2024/11/23-14:37:49.539601 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263827.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:49.820043 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263827.sst): second pass time = 1.65's, 79.716'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.28's, 152.527'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1816.612'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.153'MB/sec write SST data time = 0.07's, 616.106'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47096.151'MB/sec zip my value time = 1.65's, unzip length = 0.132'GB zip my value throughput = 79.716'MB/sec zip pipeline throughput = 110.232'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.02 avg-zval = 314.68 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.02 avg-zval = 314.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129427 bytes = 485360 } seq expand size = 905989 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9376 value = 3.2319 dict = 1.00 all = 3.1347 } Zip/UnZip{ index = 0.1441 value = 0.3094 dict = 1.00 all = 0.3190 } ---------------------------- total value len = 9.776370 GB avg = 0.184 KB (by entry num) total key len = 1.458941 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456450 Billion total entry num = 0.053264179 Billion write speed all = 0.546528211 MB/sec (with seq num) write speed all = 0.525800435 MB/sec (without seq num) 2024/11/23-14:37:49.917128 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263827: 129427 keys, 42816948 bytes, temperature: kWarm 2024/11/23-14:37:49.917214 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343869917168, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263827, "file_size": 42816948, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4030474158, "table_properties": {"data_size": 40727968, "index_size": 223872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629753, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343867, "oldest_key_time": 0, "file_creation_time": 1732343867, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263827, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:49.992671 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263828.sst) /dev/shm/ToplingTemp/Topling-76711-StrgjH 2024/11/23-14:37:50.584031 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263828.sst): first pass time = 0.59's, 226.982'MB/sec 2024/11/23-14:37:51.675511 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263828.sst): old prealloc_size = 147639500, real_size = 40871216 2024/11/23-14:37:51.675634 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263828.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.97 avg-zval = 313.78 2024/11/23-14:37:51.675809 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263828.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:51.846550 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263828.sst): second pass time = 1.09's, 120.759'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 249.706'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2184.578'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.262'MB/sec write SST data time = 0.05's, 899.875'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 37072.389'MB/sec zip my value time = 1.09's, unzip length = 0.132'GB zip my value throughput = 120.759'MB/sec zip pipeline throughput = 125.315'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.97 avg-zval = 313.78 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.97 avg-zval = 313.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129434 bytes = 485392 } seq expand size = 906038 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9004 value = 3.2410 dict = 1.00 all = 3.1430 } Zip/UnZip{ index = 0.1449 value = 0.3085 dict = 1.00 all = 0.3182 } ---------------------------- total value len = 9.908906 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585884 Billion total entry num = 0.053393613 Billion write speed all = 0.553046669 MB/sec (with seq num) write speed all = 0.532270572 MB/sec (without seq num) 2024/11/23-14:37:51.900354 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263828: 129434 keys, 42704144 bytes, temperature: kWarm 2024/11/23-14:37:51.900432 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343871900384, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263828, "file_size": 42704144, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874784, "largest_seqno": 4030479974, "table_properties": {"data_size": 40613600, "index_size": 225088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629932, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343869, "oldest_key_time": 0, "file_creation_time": 1732343869, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263828, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:51.939787 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263829.sst) /dev/shm/ToplingTemp/Topling-76711-ZGa4kL 2024/11/23-14:37:52.505963 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263829.sst): first pass time = 0.57's, 237.079'MB/sec 2024/11/23-14:37:53.478689 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263829.sst): old prealloc_size = 147639500, real_size = 40593648 2024/11/23-14:37:53.478818 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263829.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.56 2024/11/23-14:37:53.478993 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263829.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:53.643060 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263829.sst): second pass time = 0.97's, 135.429'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.113'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2208.961'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.04's, 955.345'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 22363.062'MB/sec zip my value time = 0.97's, unzip length = 0.132'GB zip my value throughput = 135.429'MB/sec zip pipeline throughput = 189.594'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1016.99 avg-zval = 311.56 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1023.99 avg-zval = 311.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5786 value = 3.2642 dict = 1.00 all = 3.1636 } Zip/UnZip{ index = 0.1520 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.041442 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715315 Billion total entry num = 0.053523044 Billion write speed all = 0.559570084 MB/sec (with seq num) write speed all = 0.538745443 MB/sec (without seq num) 2024/11/23-14:37:53.695247 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263829: 129431 keys, 42426280 bytes, temperature: kWarm 2024/11/23-14:37:53.695307 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343873695273, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263829, "file_size": 42426280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873877, "largest_seqno": 4030479280, "table_properties": {"data_size": 40325024, "index_size": 236096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629651, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343871, "oldest_key_time": 0, "file_creation_time": 1732343871, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263829, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:53.733605 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263830.sst) /dev/shm/ToplingTemp/Topling-76711-hrkfAK 2024/11/23-14:37:54.374487 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263830.sst): first pass time = 0.64's, 209.441'MB/sec 2024/11/23-14:37:55.196678 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263830.sst): old prealloc_size = 147639500, real_size = 40595968 2024/11/23-14:37:55.196806 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263830.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.83 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.58 2024/11/23-14:37:55.196991 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263830.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:55.360469 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263830.sst): second pass time = 0.82's, 160.242'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.055'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2252.842'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 940.900'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46389.212'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.242'MB/sec zip pipeline throughput = 160.242'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.83 avg-val = 1017.00 avg-zval = 311.58 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.83 avg-val = 1024.00 avg-zval = 311.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5749 value = 3.2640 dict = 1.00 all = 3.1634 } Zip/UnZip{ index = 0.1521 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.173977 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844744 Billion total entry num = 0.053652473 Billion write speed all = 0.566094526 MB/sec (with seq num) write speed all = 0.545221271 MB/sec (without seq num) 2024/11/23-14:37:55.413460 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263830: 129429 keys, 42428557 bytes, temperature: kWarm 2024/11/23-14:37:55.413529 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343875413494, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263830, "file_size": 42428557, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4030479534, "table_properties": {"data_size": 40327216, "index_size": 236224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629480, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343873, "oldest_key_time": 0, "file_creation_time": 1732343873, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263830, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:55.451679 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263831.sst) /dev/shm/ToplingTemp/Topling-76711-sDSGEH 2024/11/23-14:37:56.027045 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263831.sst): first pass time = 0.58's, 233.292'MB/sec 2024/11/23-14:37:56.909701 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263831.sst): old prealloc_size = 147639500, real_size = 41078272 2024/11/23-14:37:56.909817 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263831.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.00 avg-zval = 315.40 2024/11/23-14:37:56.909968 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263831.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:57.070394 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263831.sst): second pass time = 0.88's, 149.399'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.040'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2333.225'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 1027.207'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42637.686'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 149.399'MB/sec zip pipeline throughput = 149.399'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 315.40 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1024.00 avg-zval = 315.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9636 value = 3.2244 dict = 1.00 all = 3.1279 } Zip/UnZip{ index = 0.1436 value = 0.3101 dict = 1.00 all = 0.3197 } ---------------------------- total value len = 10.306513 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974174 Billion total entry num = 0.053781903 Billion write speed all = 0.572618099 MB/sec (with seq num) write speed all = 0.551696229 MB/sec (without seq num) 2024/11/23-14:37:57.121285 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263831: 129430 keys, 42910636 bytes, temperature: kWarm 2024/11/23-14:37:57.121354 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343877121318, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263831, "file_size": 42910636, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237916889, "largest_seqno": 4030479378, "table_properties": {"data_size": 40822704, "index_size": 223040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629707, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343875, "oldest_key_time": 0, "file_creation_time": 1732343875, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263831, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:57.159659 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263832.sst) /dev/shm/ToplingTemp/Topling-76711-VQoe2K 2024/11/23-14:37:57.884857 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263832.sst): first pass time = 0.73's, 185.089'MB/sec 2024/11/23-14:37:58.665490 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263832.sst): old prealloc_size = 147639500, real_size = 40603904 2024/11/23-14:37:58.665615 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263832.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.65 2024/11/23-14:37:58.665810 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263832.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:58.829044 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263832.sst): second pass time = 0.78's, 168.787'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.512'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2140.647'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.04's, 975.583'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46729.141'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 168.787'MB/sec zip pipeline throughput = 168.787'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1017.02 avg-zval = 311.65 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1023.76 avg-zval = 311.65 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 124590 bytes = 467232 } seq expand size = 872130 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5980 value = 3.2633 dict = 1.00 all = 3.1626 } Zip/UnZip{ index = 0.1516 value = 0.3064 dict = 1.00 all = 0.3162 } ---------------------------- total value len = 10.439015 GB avg = 0.194 KB (by entry num) total key len = 1.471884 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103601 Billion total entry num = 0.053911330 Billion write speed all = 0.579137568 MB/sec (with seq num) write speed all = 0.558167143 MB/sec (without seq num) 2024/11/23-14:37:58.881074 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263832: 129427 keys, 42439139 bytes, temperature: kWarm 2024/11/23-14:37:58.881145 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343878881108, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263832, "file_size": 42439139, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4030475692, "table_properties": {"data_size": 40335984, "index_size": 235392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629764, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343877, "oldest_key_time": 0, "file_creation_time": 1732343877, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263832, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:58.918605 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263833.sst) /dev/shm/ToplingTemp/Topling-76711-YjPT8J 2024/11/23-14:37:59.558680 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263833.sst): first pass time = 0.64's, 209.703'MB/sec 2024/11/23-14:38:00.385784 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263833.sst): old prealloc_size = 147639500, real_size = 40536032 2024/11/23-14:38:00.385901 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263833.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.23 2024/11/23-14:38:00.385999 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263833.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:00.543304 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263833.sst): second pass time = 0.83's, 159.414'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.798'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2344.093'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.279'MB/sec write SST data time = 0.04's, 1012.851'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 40928.457'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 159.414'MB/sec zip pipeline throughput = 159.414'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1016.99 avg-zval = 311.23 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1018.66 avg-zval = 311.23 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 30988 bytes = 116224 } seq expand size = 216916 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 7.0425 value = 3.2676 dict = 1.00 all = 3.1942 } Zip/UnZip{ index = 0.1420 value = 0.3060 dict = 1.00 all = 0.3131 } ---------------------------- total value len = 10.570861 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233032 Billion total entry num = 0.054040761 Billion write speed all = 0.585625331 MB/sec (with seq num) write speed all = 0.564606312 MB/sec (without seq num) 2024/11/23-14:38:00.593982 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263833: 129431 keys, 42019489 bytes, temperature: kWarm 2024/11/23-14:38:00.594051 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343880594017, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263833, "file_size": 42019489, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4030479251, "table_properties": {"data_size": 40282960, "index_size": 220544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629711, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343878, "oldest_key_time": 0, "file_creation_time": 1732343878, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263833, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:00.632793 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263834.sst) /dev/shm/ToplingTemp/Topling-76711-sJOHMI 2024/11/23-14:38:01.241024 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263834.sst): first pass time = 0.61's, 220.685'MB/sec 2024/11/23-14:38:02.161076 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263834.sst): old prealloc_size = 147639500, real_size = 41009216 2024/11/23-14:38:02.161201 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263834.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.01 avg-zval = 314.88 2024/11/23-14:38:02.161350 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263834.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:02.321495 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263834.sst): second pass time = 0.92's, 143.188'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.857'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2161.369'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 1014.269'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44406.142'MB/sec zip my value time = 0.92's, unzip length = 0.132'GB zip my value throughput = 143.188'MB/sec zip pipeline throughput = 143.188'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.01 avg-zval = 314.88 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1023.17 avg-zval = 314.88 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 113879 bytes = 427056 } seq expand size = 797153 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9695 value = 3.2299 dict = 1.00 all = 3.1356 } Zip/UnZip{ index = 0.1435 value = 0.3096 dict = 1.00 all = 0.3189 } ---------------------------- total value len = 10.703288 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362460 Billion total entry num = 0.054170189 Billion write speed all = 0.592138373 MB/sec (with seq num) write speed all = 0.571070835 MB/sec (without seq num) 2024/11/23-14:38:02.374315 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263834: 129428 keys, 42803950 bytes, temperature: kWarm 2024/11/23-14:38:02.374381 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343882374348, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263834, "file_size": 42803950, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4030458552, "table_properties": {"data_size": 40753840, "index_size": 222848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629617, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343880, "oldest_key_time": 0, "file_creation_time": 1732343880, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263834, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:02.411160 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263835.sst) /dev/shm/ToplingTemp/Topling-76711-wjd8dH 2024/11/23-14:38:03.062863 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263835.sst): first pass time = 0.65's, 205.962'MB/sec 2024/11/23-14:38:03.875813 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263835.sst): old prealloc_size = 147639500, real_size = 40905824 2024/11/23-14:38:03.875939 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263835.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.02 avg-zval = 314.08 2024/11/23-14:38:03.876087 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263835.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:04.037386 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263835.sst): second pass time = 0.81's, 162.060'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.828'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2154.010'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 975.015'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48098.956'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.060'MB/sec zip pipeline throughput = 162.060'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.02 avg-zval = 314.08 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1022.04 avg-zval = 314.08 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 92835 bytes = 348144 } seq expand size = 649845 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.9815 value = 3.2381 dict = 1.00 all = 3.1491 } Zip/UnZip{ index = 0.1432 value = 0.3088 dict = 1.00 all = 0.3176 } ---------------------------- total value len = 10.835568 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021175 GB avg = 0.015 KB total ukey num = 0.067491887 Billion total entry num = 0.054299616 Billion write speed all = 0.598644946 MB/sec (with seq num) write speed all = 0.577528833 MB/sec (without seq num) 2024/11/23-14:38:04.089972 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263835: 129427 keys, 42621634 bytes, temperature: kWarm 2024/11/23-14:38:04.090054 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343884090017, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263835, "file_size": 42621634, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4030450097, "table_properties": {"data_size": 40650832, "index_size": 222464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629713, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343882, "oldest_key_time": 0, "file_creation_time": 1732343882, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263835, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:04.128692 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263836.sst) /dev/shm/ToplingTemp/Topling-76711-CWILoI 2024/11/23-14:38:04.939282 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263836.sst): first pass time = 0.81's, 165.588'MB/sec 2024/11/23-14:38:05.788398 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263836.sst): old prealloc_size = 147639500, real_size = 40745808 2024/11/23-14:38:05.788529 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263836.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.00 avg-zval = 312.79 2024/11/23-14:38:05.788681 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263836.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:05.949478 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263836.sst): second pass time = 0.85's, 155.130'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.803'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2279.567'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 974.280'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44797.835'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 155.130'MB/sec zip pipeline throughput = 155.130'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1017.00 avg-zval = 312.79 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1022.39 avg-zval = 312.79 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 99756 bytes = 374096 } seq expand size = 698292 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.7864 value = 3.2514 dict = 1.00 all = 3.1590 } Zip/UnZip{ index = 0.1474 value = 0.3076 dict = 1.00 all = 0.3166 } ---------------------------- total value len = 10.967896 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621317 Billion total entry num = 0.054429046 Billion write speed all = 0.605147031 MB/sec (with seq num) write speed all = 0.583982552 MB/sec (without seq num) 2024/11/23-14:38:06.001310 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263836: 129430 keys, 42487542 bytes, temperature: kWarm 2024/11/23-14:38:06.001380 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343886001345, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263836, "file_size": 42487542, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4030479379, "table_properties": {"data_size": 40484416, "index_size": 228864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629940, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343884, "oldest_key_time": 0, "file_creation_time": 1732343884, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263836, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:06.039648 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263837.sst) /dev/shm/ToplingTemp/Topling-76711-tacf3I 2024/11/23-14:38:06.547089 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263837.sst): first pass time = 0.51's, 264.526'MB/sec 2024/11/23-14:38:07.348867 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263837.sst): old prealloc_size = 147639500, real_size = 40454928 2024/11/23-14:38:07.348998 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263837.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.60 2024/11/23-14:38:07.349088 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263837.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:07.507895 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263837.sst): second pass time = 0.80's, 164.549'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.637'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1998.164'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 999.432'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45559.057'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 164.549'MB/sec zip pipeline throughput = 164.549'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 310.60 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.43 avg-zval = 310.60 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 26285 bytes = 98576 } seq expand size = 183995 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.9957 value = 3.2743 dict = 1.00 all = 3.2017 } Zip/UnZip{ index = 0.1429 value = 0.3054 dict = 1.00 all = 0.3123 } ---------------------------- total value len = 11.099710 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750746 Billion total entry num = 0.054558475 Billion write speed all = 0.611633420 MB/sec (with seq num) write speed all = 0.590420220 MB/sec (without seq num) 2024/11/23-14:38:07.558359 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263837: 129429 keys, 41920964 bytes, temperature: kWarm 2024/11/23-14:38:07.558427 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343887558393, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263837, "file_size": 41920964, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4030468004, "table_properties": {"data_size": 40200384, "index_size": 222016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629859, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343886, "oldest_key_time": 0, "file_creation_time": 1732343886, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263837, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:07.595574 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263838.sst) /dev/shm/ToplingTemp/Topling-76711-IbL3HI 2024/11/23-14:38:08.289894 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263838.sst): first pass time = 0.69's, 193.319'MB/sec 2024/11/23-14:38:09.270735 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263838.sst): old prealloc_size = 147639500, real_size = 40584336 2024/11/23-14:38:09.270858 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263838.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.60 2024/11/23-14:38:09.270940 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263838.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:09.433173 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263838.sst): second pass time = 0.98's, 134.461'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.903'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2233.104'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 971.760'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.379'MB, throughput = 37703.851'MB/sec zip my value time = 0.98's, unzip length = 0.132'GB zip my value throughput = 134.461'MB/sec zip pipeline throughput = 134.461'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.02 avg-zval = 311.60 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.43 avg-zval = 311.60 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 26043 bytes = 97680 } seq expand size = 182301 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9935 value = 3.2638 dict = 1.00 all = 3.1918 } Zip/UnZip{ index = 0.1430 value = 0.3064 dict = 1.00 all = 0.3133 } ---------------------------- total value len = 11.231522 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880173 Billion total entry num = 0.054687902 Billion write speed all = 0.618107719 MB/sec (with seq num) write speed all = 0.596846186 MB/sec (without seq num) 2024/11/23-14:38:09.488854 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263838: 129427 keys, 42050358 bytes, temperature: kWarm 2024/11/23-14:38:09.488926 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343889488890, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263838, "file_size": 42050358, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4030413459, "table_properties": {"data_size": 40329728, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629800, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343887, "oldest_key_time": 0, "file_creation_time": 1732343887, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263838, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:09.526632 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263839.sst) /dev/shm/ToplingTemp/Topling-76711-Zve8RI 2024/11/23-14:38:10.177465 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263839.sst): first pass time = 0.65's, 206.241'MB/sec 2024/11/23-14:38:10.984731 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263839.sst): old prealloc_size = 147639500, real_size = 40453248 2024/11/23-14:38:10.984849 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263839.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.58 2024/11/23-14:38:10.984930 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263839.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:11.144788 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263839.sst): second pass time = 0.81's, 163.354'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.917'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2229.562'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 970.102'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47256.912'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 163.354'MB/sec zip pipeline throughput = 163.354'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 310.58 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.36 avg-zval = 310.58 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 25326 bytes = 94992 } seq expand size = 177282 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.9937 value = 3.2745 dict = 1.00 all = 3.2021 } Zip/UnZip{ index = 0.1430 value = 0.3054 dict = 1.00 all = 0.3123 } ---------------------------- total value len = 11.363329 GB avg = 0.207 KB (by entry num) total key len = 1.490004 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009603 Billion total entry num = 0.054817332 Billion write speed all = 0.624587036 MB/sec (with seq num) write speed all = 0.603276956 MB/sec (without seq num) 2024/11/23-14:38:11.196609 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263839: 129430 keys, 41915647 bytes, temperature: kWarm 2024/11/23-14:38:11.196685 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343891196643, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263839, "file_size": 41915647, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4030478875, "table_properties": {"data_size": 40198640, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629482, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343889, "oldest_key_time": 0, "file_creation_time": 1732343889, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263839, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:11.237399 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263840.sst) /dev/shm/ToplingTemp/Topling-76711-3EPsWG 2024/11/23-14:38:11.750231 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263840.sst): first pass time = 0.51's, 261.740'MB/sec 2024/11/23-14:38:12.738331 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263840.sst): old prealloc_size = 147639500, real_size = 40758480 2024/11/23-14:38:12.738453 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263840.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.00 avg-zval = 312.94 2024/11/23-14:38:12.738584 139957685155392 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263840.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:12.898968 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263840.sst): second pass time = 0.99's, 133.379'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.384'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2257.648'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 996.875'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 37271.284'MB/sec zip my value time = 0.99's, unzip length = 0.132'GB zip my value throughput = 133.379'MB/sec zip pipeline throughput = 206.775'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 312.94 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1021.87 avg-zval = 312.94 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 90081 bytes = 337824 } seq expand size = 630567 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.9755 value = 3.2498 dict = 1.00 all = 3.1607 } Zip/UnZip{ index = 0.1434 value = 0.3077 dict = 1.00 all = 0.3164 } ---------------------------- total value len = 11.495589 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139032 Billion total entry num = 0.054946761 Billion write speed all = 0.631085990 MB/sec (with seq num) write speed all = 0.609727415 MB/sec (without seq num) 2024/11/23-14:38:12.952044 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263840: 129429 keys, 42464158 bytes, temperature: kWarm 2024/11/23-14:38:12.952109 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343892952077, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263840, "file_size": 42464158, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4030479905, "table_properties": {"data_size": 40503296, "index_size": 222656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629417, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343891, "oldest_key_time": 0, "file_creation_time": 1732343891, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263840, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:12.990655 139957685155392 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263841.sst) /dev/shm/ToplingTemp/Topling-76711-7j1RLK 2024/11/23-14:38:13.000797 139957685155392 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263841.sst): first pass time = 0.01's, 461.094'MB/sec 2024/11/23-14:38:13.038825 139957685155392 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263841.sst): old prealloc_size = 147639500, real_size = 1504432 2024/11/23-14:38:13.038888 139957685155392 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263841.sst): index type = FixedLenKeyIndex , store type = DictZipBlobStore usrkeys = 4491 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 7.54 raw-val = 0.0046 GB zip-val = 0.0015 GB avg-val =1016.98 avg-zval = 327.16 2024/11/23-14:38:13.046643 139957685155392 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00/263841.sst): second pass time = 0.04's, 121.902'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.01's, 200.798'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 762.729'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.184'MB/sec write SST data time = 0.00's, 940.434'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.047'MB, throughput = 1955.225'MB/sec zip my value time = 0.04's, unzip length = 0.005'GB zip my value throughput = 121.902'MB/sec zip pipeline throughput = 121.902'MB/sec entries = 4491 avg-key = 20.00 avg-zkey = 7.54 avg-val = 1016.98 avg-zval = 327.16 usrkeys = 4491 avg-key = 12.00 avg-zkey = 7.54 avg-val = 1023.98 avg-zval = 327.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 4491 bytes = 16864 } seq expand size = 31437 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0001 GB value = 0.0046 GB dict = 0.05 MB all = 0.0047 GB } __ZipSize{ index = 0.0000 GB value = 0.0015 GB dict = 0.05 MB all = 0.0016 GB } UnZip/Zip{ index = 1.5918 value = 3.1085 dict = 1.00 all = 2.9696 } Zip/UnZip{ index = 0.6282 value = 0.3217 dict = 1.00 all = 0.3367 } ---------------------------- total value len = 11.500187 GB avg = 0.209 KB (by entry num) total key len = 1.492683 GB avg = 0.027 KB total ukey len = 1.028995 GB avg = 0.015 KB total ukey num = 0.068143523 Billion total entry num = 0.054951252 Billion write speed all = 0.631309271 MB/sec (with seq num) write speed all = 0.609949104 MB/sec (without seq num) 2024/11/23-14:38:13.054812 139957685155392 [compaction_job.cc:1948] [default] [JOB 28061] Generated table #263841: 4491 keys, 1568235 bytes, temperature: kWarm 2024/11/23-14:38:13.054861 139957685155392 EVENT_LOG_v1 {"time_micros": 1732343893054833, "cf_name": "default", "job": 28061, "event": "table_file_creation", "file_number": 263841, "file_size": 1568235, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3356359572, "largest_seqno": 4028963389, "table_properties": {"data_size": 1469280, "index_size": 33856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 89820, "raw_average_key_size": 20, "raw_value_size": 4567251, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 4491, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-FixedLenKeyIndex-DictZipBlobStore", "creation_time": 1732343892, "oldest_key_time": 0, "file_creation_time": 1732343892, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263841, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:13.066488 139957685155392 [compaction_job.cc:703] job-28061: subcompact[0], size: 0.851147 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28061/att-00 [263821,263822,263823,263824,263825,263826,263827,263828,263829,263830,263831,263832,263833,263834,263835,263836,263837,263838,263839,263840,263841] 2024/11/23-14:38:13.199520 139957685155392 (Original Log Time 2024/11/23-14:37:33.994047) EVENT_LOG_v1 {"time_micros": 1732343853993998, "job": 28061, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [263816], "files_L3": [263737, 263738, 263739, 263740, 263741, 263742, 263743, 263744, 263745, 263746, 263747, 263748, 263749, 263750, 263751, 263752, 263753, 263754, 263755, 263758], "score": 0.998312, "input_data_size": 882290006, "oldest_snapshot_seqno": 4030762260} 2024/11/23-14:38:13.199524 139957685155392 (Original Log Time 2024/11/23-14:38:13.197863) [compaction_job.cc:2022] [default] [JOB 28061] Compacted 1@2 + 20@3 files to L3 => 851147356 bytes 2024/11/23-14:38:13.199525 139957685155392 (Original Log Time 2024/11/23-14:38:13.199445) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 22.6 rd, 21.8 wr, level 3, files in(1, 20) out(21 +0 blob) MB in(31.2, 810.2 +0.0 blob) out(811.7 +0.0 blob), read-write-amplify(53.0) write-amplify(26.0) OK, records in: 2620507, records dropped: 27419 output_compression: Snappy 2024/11/23-14:38:13.199529 139957685155392 (Original Log Time 2024/11/23-14:38:13.199494) EVENT_LOG_v1 {"time_micros": 1732343893199471, "cf": "default", "job": 28061, "event": "compaction_finished", "compaction_time_micros": 39072445, "compaction_time_cpu_micros": 7918009, "output_level": 3, "num_output_files": 21, "total_output_size": 851147356, "num_input_records": 2620507, "num_output_records": 2593088, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 606186730, "file_range_sync_nanos": 0, "file_fsync_nanos": 15010, "file_prepare_write_nanos": 45977, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/23-14:38:13.345216 139957685155392 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732343853,"executesMs":39361,"compactionJobId":28061,"attempt":0,"compactionInputRawBytes":2717456312,"compactionInputZipBytes":882290006,"compactionOutputRawBytes":0,"compactionOutputZipBytes":851147356,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-689000c6af7816a855b6e5259cc924d3-a7623f3431834ad9-00"}}, response = 2024/11/23-14:38:13.345307 139957685155392 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28061/att-00: olev 3, work 39.361 s, result 7.791 ms, install 1.682 ms, input{raw 2.531 GiB zip 841.417 MiB}