2024/11/23-14:39:12.211566 139957559264832 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00: fork to child time = 0.008209 sec 2024/11/23-14:39:12.212273 139957559264832 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00 2024/11/23-14:39:12.219388 139957559264832 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00 2024/11/23-14:39:12.220271 139957559264832 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:39:12.220520 139957559264832 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:39:12.280764 139957559264832 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28109/att-00/MANIFEST-206347 2024/11/23-14:39:12.296328 139957559264832 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28109/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 263899, last_sequence is 4032018301, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 263764 2024/11/23-14:39:12.296333 139957559264832 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:39:12.296441 139957559264832 [version_set.cc:5721] Creating manifest 263899 2024/11/23-14:39:12.335361 139957559264832 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:39:12.358916 139957559264832 [compaction_job.cc:2437] [default] [JOB 28109] Compacting 1@2 + 20@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-14:39:12.358932 139957559264832 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20997 Base level 2, inputs: [263896(31MB)], [263851(40MB) 263852(40MB) 263853(40MB) 263854(40MB) 263855(40MB) 263856(40MB) 263857(40MB) 263858(40MB) 263859(40MB) 263860(40MB) 263861(40MB) 263862(40MB) 263863(40MB) 263864(40MB) 263865(40MB) 263866(40MB) 263867(39MB) 263868(40MB) 263869(39MB) 263872(40MB)] 2024/11/23-14:39:12.366113 139957559264832 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:39:12.366144 139957559264832 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:39:12.366163 139957559264832 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:39:12.366169 139957559264832 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:39:12.366180 139957559264832 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:39:13.077799 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263900.sst) /dev/shm/ToplingTemp/Topling-77275-OdqbCm 2024/11/23-14:39:13.297523 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263900.sst): first pass time = 0.22's, 610.961'MB/sec 2024/11/23-14:39:14.419285 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263900.sst): old prealloc_size = 147639500, real_size = 40692272 2024/11/23-14:39:14.419423 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263900.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.99 avg-zval = 312.34 2024/11/23-14:39:14.419808 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263900.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:14.594718 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263900.sst): second pass time = 1.12's, 117.437'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 242.422'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2138.274'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.256'MB/sec write SST data time = 0.05's, 870.110'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39855.271'MB/sec zip my value time = 1.12's, unzip length = 0.132'GB zip my value throughput = 117.437'MB/sec zip pipeline throughput = 150.048'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.99 avg-zval = 312.34 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.99 avg-zval = 312.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6489 value = 3.2561 dict = 1.00 all = 3.1562 } Zip/UnZip{ index = 0.1504 value = 0.3071 dict = 1.00 all = 0.3168 } ---------------------------- total value len = 8.981155 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679866 Billion total entry num = 0.052487595 Billion write speed all = 0.505007902 MB/sec (with seq num) write speed all = 0.484666219 MB/sec (without seq num) 2024/11/23-14:39:14.668202 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263900: 129431 keys, 42525067 bytes, temperature: kWarm 2024/11/23-14:39:14.668286 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343954668233, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263900, "file_size": 42525067, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4031987341, "table_properties": {"data_size": 40426144, "index_size": 233600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629547, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343953, "oldest_key_time": 0, "file_creation_time": 1732343953, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263900, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:14.718878 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263901.sst) /dev/shm/ToplingTemp/Topling-77275-EZ15Xm 2024/11/23-14:39:15.435552 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263901.sst): first pass time = 0.72's, 187.289'MB/sec 2024/11/23-14:39:17.240526 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263901.sst): old prealloc_size = 147639500, real_size = 40821440 2024/11/23-14:39:17.240646 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263901.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.99 avg-zval = 313.37 2024/11/23-14:39:17.240812 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263901.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:17.410909 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263901.sst): second pass time = 1.79's, 73.568'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 250.373'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2187.605'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.265'MB/sec write SST data time = 0.05's, 883.811'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39738.662'MB/sec zip my value time = 1.79's, unzip length = 0.132'GB zip my value throughput = 73.568'MB/sec zip pipeline throughput = 115.478'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1016.99 avg-zval = 313.37 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1023.99 avg-zval = 313.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.7978 value = 3.2453 dict = 1.00 all = 3.1466 } Zip/UnZip{ index = 0.1471 value = 0.3081 dict = 1.00 all = 0.3178 } ---------------------------- total value len = 9.113691 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809297 Billion total entry num = 0.052617026 Billion write speed all = 0.511484087 MB/sec (with seq num) write speed all = 0.491095024 MB/sec (without seq num) 2024/11/23-14:39:17.491846 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263901: 129431 keys, 42654479 bytes, temperature: kWarm 2024/11/23-14:39:17.491895 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343957491867, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263901, "file_size": 42654479, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4031990378, "table_properties": {"data_size": 40560432, "index_size": 228480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629662, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343954, "oldest_key_time": 0, "file_creation_time": 1732343954, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263901, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:17.565746 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263902.sst) /dev/shm/ToplingTemp/Topling-77275-7BTcip 2024/11/23-14:39:18.269421 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263902.sst): first pass time = 0.70's, 190.748'MB/sec 2024/11/23-14:39:19.506321 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263902.sst): old prealloc_size = 147639500, real_size = 40717840 2024/11/23-14:39:19.506443 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263902.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.97 avg-zval = 312.54 2024/11/23-14:39:19.506609 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263902.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:19.676873 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263902.sst): second pass time = 1.22's, 107.563'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 249.504'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2300.652'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.263'MB/sec write SST data time = 0.05's, 899.718'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46083.715'MB/sec zip my value time = 1.22's, unzip length = 0.132'GB zip my value throughput = 107.563'MB/sec zip pipeline throughput = 119.870'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.97 avg-zval = 312.54 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.97 avg-zval = 312.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.6746 value = 3.2539 dict = 1.00 all = 3.1543 } Zip/UnZip{ index = 0.1498 value = 0.3073 dict = 1.00 all = 0.3170 } ---------------------------- total value len = 9.246226 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938730 Billion total entry num = 0.052746459 Billion write speed all = 0.517972298 MB/sec (with seq num) write speed all = 0.497535323 MB/sec (without seq num) 2024/11/23-14:39:19.729191 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263902: 129433 keys, 42550464 bytes, temperature: kWarm 2024/11/23-14:39:19.729260 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343959729228, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263902, "file_size": 42550464, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237870807, "largest_seqno": 4031990250, "table_properties": {"data_size": 40452608, "index_size": 232704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629408, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343957, "oldest_key_time": 0, "file_creation_time": 1732343957, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263902, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:19.800245 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263903.sst) /dev/shm/ToplingTemp/Topling-77275-OBzcFm 2024/11/23-14:39:20.344539 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263903.sst): first pass time = 0.54's, 246.615'MB/sec 2024/11/23-14:39:21.288539 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263903.sst): old prealloc_size = 147639500, real_size = 40795776 2024/11/23-14:39:21.288724 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263903.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.96 avg-zval = 313.16 2024/11/23-14:39:21.288909 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263903.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:21.477879 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263903.sst): second pass time = 0.94's, 140.135'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 225.136'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1402.539'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.235'MB/sec write SST data time = 0.05's, 829.159'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39352.269'MB/sec zip my value time = 0.94's, unzip length = 0.132'GB zip my value throughput = 140.135'MB/sec zip pipeline throughput = 140.135'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1016.96 avg-zval = 313.16 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1023.96 avg-zval = 313.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129435 bytes = 485392 } seq expand size = 906045 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7696 value = 3.2474 dict = 1.00 all = 3.1486 } Zip/UnZip{ index = 0.1477 value = 0.3079 dict = 1.00 all = 0.3176 } ---------------------------- total value len = 9.378762 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068165 Billion total entry num = 0.052875894 Billion write speed all = 0.524470912 MB/sec (with seq num) write speed all = 0.503985574 MB/sec (without seq num) 2024/11/23-14:39:21.530690 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263903: 129435 keys, 42628400 bytes, temperature: kWarm 2024/11/23-14:39:21.530937 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343961530715, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263903, "file_size": 42628400, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4031989534, "table_properties": {"data_size": 40533808, "index_size": 229440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629729, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343959, "oldest_key_time": 0, "file_creation_time": 1732343959, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263903, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:21.579005 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263904.sst) /dev/shm/ToplingTemp/Topling-77275-QlvkBm 2024/11/23-14:39:22.232392 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263904.sst): first pass time = 0.65's, 205.429'MB/sec 2024/11/23-14:39:23.117885 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263904.sst): old prealloc_size = 147639500, real_size = 40594880 2024/11/23-14:39:23.118028 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263904.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.83 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.57 2024/11/23-14:39:23.118185 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263904.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:23.295270 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263904.sst): second pass time = 0.88's, 148.805'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 239.239'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1953.088'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.06's, 751.432'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 54767.451'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 148.805'MB/sec zip pipeline throughput = 148.805'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.83 avg-val = 1017.00 avg-zval = 311.57 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.83 avg-val = 1024.00 avg-zval = 311.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5749 value = 3.2641 dict = 1.00 all = 3.1635 } Zip/UnZip{ index = 0.1521 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.511297 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197594 Billion total entry num = 0.053005323 Billion write speed all = 0.530967938 MB/sec (with seq num) write speed all = 0.510434263 MB/sec (without seq num) 2024/11/23-14:39:23.351775 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263904: 129429 keys, 42427393 bytes, temperature: kWarm 2024/11/23-14:39:23.351864 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343963351817, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263904, "file_size": 42427393, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4031985160, "table_properties": {"data_size": 40326128, "index_size": 236224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629178, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343961, "oldest_key_time": 0, "file_creation_time": 1732343961, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263904, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:23.421485 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263905.sst) /dev/shm/ToplingTemp/Topling-77275-tYa9nl 2024/11/23-14:39:24.364378 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263905.sst): first pass time = 0.94's, 142.356'MB/sec 2024/11/23-14:39:25.788099 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263905.sst): old prealloc_size = 147639500, real_size = 40958304 2024/11/23-14:39:25.788247 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263905.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.01 avg-zval = 314.45 2024/11/23-14:39:25.788432 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263905.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:25.981164 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263905.sst): second pass time = 1.40's, 93.875'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 221.648'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1852.540'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.231'MB/sec write SST data time = 0.05's, 808.453'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45341.603'MB/sec zip my value time = 1.40's, unzip length = 0.132'GB zip my value throughput = 93.875'MB/sec zip pipeline throughput = 128.814'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1017.01 avg-zval = 314.45 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1024.01 avg-zval = 314.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129428 bytes = 485376 } seq expand size = 905996 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.8341 value = 3.2343 dict = 1.00 all = 3.1366 } Zip/UnZip{ index = 0.1463 value = 0.3092 dict = 1.00 all = 0.3188 } ---------------------------- total value len = 9.643833 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327022 Billion total entry num = 0.053134751 Billion write speed all = 0.537441233 MB/sec (with seq num) write speed all = 0.516860096 MB/sec (without seq num) 2024/11/23-14:39:26.033936 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263905: 129428 keys, 42791077 bytes, temperature: kWarm 2024/11/23-14:39:26.034033 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343966033969, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263905, "file_size": 42791077, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237885133, "largest_seqno": 4031982185, "table_properties": {"data_size": 40698512, "index_size": 227264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629506, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343963, "oldest_key_time": 0, "file_creation_time": 1732343963, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263905, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:26.093413 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263906.sst) /dev/shm/ToplingTemp/Topling-77275-tiTXPm 2024/11/23-14:39:26.643157 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263906.sst): first pass time = 0.55's, 244.166'MB/sec 2024/11/23-14:39:27.514632 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263906.sst): old prealloc_size = 147639500, real_size = 40984224 2024/11/23-14:39:27.514763 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263906.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.02 avg-zval = 314.68 2024/11/23-14:39:27.514923 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263906.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:27.697145 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263906.sst): second pass time = 0.87's, 151.353'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 234.624'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2041.920'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.238'MB/sec write SST data time = 0.05's, 922.296'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49463.038'MB/sec zip my value time = 0.87's, unzip length = 0.132'GB zip my value throughput = 151.353'MB/sec zip pipeline throughput = 151.353'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.02 avg-zval = 314.68 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.02 avg-zval = 314.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129427 bytes = 485360 } seq expand size = 905989 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9376 value = 3.2319 dict = 1.00 all = 3.1347 } Zip/UnZip{ index = 0.1441 value = 0.3094 dict = 1.00 all = 0.3190 } ---------------------------- total value len = 9.776369 GB avg = 0.184 KB (by entry num) total key len = 1.458941 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456449 Billion total entry num = 0.053264178 Billion write speed all = 0.543938414 MB/sec (with seq num) write speed all = 0.523308858 MB/sec (without seq num) 2024/11/23-14:39:27.748987 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263906: 129427 keys, 42816605 bytes, temperature: kWarm 2024/11/23-14:39:27.749054 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343967749021, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263906, "file_size": 42816605, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4031987607, "table_properties": {"data_size": 40727824, "index_size": 223872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630107, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343966, "oldest_key_time": 0, "file_creation_time": 1732343966, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263906, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:27.789059 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263907.sst) /dev/shm/ToplingTemp/Topling-77275-wNNS7o 2024/11/23-14:39:28.630446 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263907.sst): first pass time = 0.84's, 159.528'MB/sec 2024/11/23-14:39:30.187129 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263907.sst): old prealloc_size = 147639500, real_size = 40884752 2024/11/23-14:39:30.187281 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263907.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.97 avg-zval = 313.89 2024/11/23-14:39:30.187468 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263907.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:30.396732 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263907.sst): second pass time = 1.56's, 84.602'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 203.824'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1738.428'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.200'MB/sec write SST data time = 0.05's, 890.065'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44898.941'MB/sec zip my value time = 1.56's, unzip length = 0.132'GB zip my value throughput = 84.602'MB/sec zip pipeline throughput = 145.311'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.97 avg-zval = 313.89 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.97 avg-zval = 313.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129434 bytes = 485392 } seq expand size = 906038 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9122 value = 3.2399 dict = 1.00 all = 3.1420 } Zip/UnZip{ index = 0.1447 value = 0.3086 dict = 1.00 all = 0.3183 } ---------------------------- total value len = 9.908905 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585883 Billion total entry num = 0.053393612 Billion write speed all = 0.550408311 MB/sec (with seq num) write speed all = 0.529731327 MB/sec (without seq num) 2024/11/23-14:39:30.449173 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263907: 129434 keys, 42717692 bytes, temperature: kWarm 2024/11/23-14:39:30.449241 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343970449206, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263907, "file_size": 42717692, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874784, "largest_seqno": 4031990703, "table_properties": {"data_size": 40627520, "index_size": 224704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629996, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343967, "oldest_key_time": 0, "file_creation_time": 1732343967, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263907, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:30.486600 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263908.sst) /dev/shm/ToplingTemp/Topling-77275-pGyG3k 2024/11/23-14:39:31.074424 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263908.sst): first pass time = 0.59's, 228.351'MB/sec 2024/11/23-14:39:31.894586 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263908.sst): old prealloc_size = 147639500, real_size = 40597504 2024/11/23-14:39:31.894738 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263908.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.59 2024/11/23-14:39:31.894923 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263908.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:32.071918 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263908.sst): second pass time = 0.82's, 160.625'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 239.281'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1855.707'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.06's, 751.314'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44818.084'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 160.625'MB/sec zip pipeline throughput = 160.625'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1016.99 avg-zval = 311.59 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1023.99 avg-zval = 311.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5786 value = 3.2639 dict = 1.00 all = 3.1632 } Zip/UnZip{ index = 0.1520 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.041441 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715314 Billion total entry num = 0.053523043 Billion write speed all = 0.556904120 MB/sec (with seq num) write speed all = 0.536178694 MB/sec (without seq num) 2024/11/23-14:39:32.125034 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263908: 129431 keys, 42430712 bytes, temperature: kWarm 2024/11/23-14:39:32.125104 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343972125067, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263908, "file_size": 42430712, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873877, "largest_seqno": 4031984575, "table_properties": {"data_size": 40328880, "index_size": 236096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629894, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343970, "oldest_key_time": 0, "file_creation_time": 1732343970, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263908, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:32.170430 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263909.sst) /dev/shm/ToplingTemp/Topling-77275-PrfO1o 2024/11/23-14:39:32.968527 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263909.sst): first pass time = 0.80's, 168.183'MB/sec 2024/11/23-14:39:34.371056 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263909.sst): old prealloc_size = 147639500, real_size = 40594320 2024/11/23-14:39:34.371195 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263909.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.83 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.56 2024/11/23-14:39:34.371367 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263909.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:34.557328 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263909.sst): second pass time = 1.40's, 93.899'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 227.882'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2142.330'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.255'MB/sec write SST data time = 0.06's, 716.649'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43487.980'MB/sec zip my value time = 1.40's, unzip length = 0.132'GB zip my value throughput = 93.899'MB/sec zip pipeline throughput = 115.840'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.83 avg-val = 1016.99 avg-zval = 311.56 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.83 avg-val = 1023.99 avg-zval = 311.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5750 value = 3.2642 dict = 1.00 all = 3.1635 } Zip/UnZip{ index = 0.1521 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.173977 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844745 Billion total entry num = 0.053652474 Billion write speed all = 0.563376786 MB/sec (with seq num) write speed all = 0.542603740 MB/sec (without seq num) 2024/11/23-14:39:34.613157 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263909: 129431 keys, 42427482 bytes, temperature: kWarm 2024/11/23-14:39:34.613221 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343974613186, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263909, "file_size": 42427482, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4031982660, "table_properties": {"data_size": 40325568, "index_size": 236224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629908, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343972, "oldest_key_time": 0, "file_creation_time": 1732343972, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263909, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:34.654032 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263910.sst) /dev/shm/ToplingTemp/Topling-77275-Vpjnwp 2024/11/23-14:39:35.328188 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263910.sst): first pass time = 0.67's, 199.100'MB/sec 2024/11/23-14:39:36.156256 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263910.sst): old prealloc_size = 147639500, real_size = 41078512 2024/11/23-14:39:36.156371 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263910.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.00 avg-zval = 315.41 2024/11/23-14:39:36.156515 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263910.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:36.318491 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263910.sst): second pass time = 0.83's, 159.280'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.533'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2356.397'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 965.452'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46557.054'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 159.280'MB/sec zip pipeline throughput = 159.280'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 315.41 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1024.00 avg-zval = 315.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9636 value = 3.2244 dict = 1.00 all = 3.1278 } Zip/UnZip{ index = 0.1436 value = 0.3101 dict = 1.00 all = 0.3197 } ---------------------------- total value len = 10.306513 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974175 Billion total entry num = 0.053781904 Billion write speed all = 0.569867859 MB/sec (with seq num) write speed all = 0.549046476 MB/sec (without seq num) 2024/11/23-14:39:36.372122 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263910: 129430 keys, 42911167 bytes, temperature: kWarm 2024/11/23-14:39:36.372191 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343976372155, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263910, "file_size": 42911167, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237916889, "largest_seqno": 4031979548, "table_properties": {"data_size": 40822944, "index_size": 223040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629835, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343974, "oldest_key_time": 0, "file_creation_time": 1732343974, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263910, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:36.412182 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263911.sst) /dev/shm/ToplingTemp/Topling-77275-7gjD0k 2024/11/23-14:39:36.956643 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263911.sst): first pass time = 0.54's, 246.532'MB/sec 2024/11/23-14:39:37.751767 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263911.sst): old prealloc_size = 147639500, real_size = 40594560 2024/11/23-14:39:37.751885 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263911.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.57 2024/11/23-14:39:37.752061 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263911.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:37.912846 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263911.sst): second pass time = 0.79's, 165.867'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.527'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2444.774'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 963.444'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48223.085'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 165.867'MB/sec zip pipeline throughput = 165.867'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1017.02 avg-zval = 311.57 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1023.95 avg-zval = 311.57 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 128191 bytes = 480736 } seq expand size = 897337 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5855 value = 3.2641 dict = 1.00 all = 3.1623 } Zip/UnZip{ index = 0.1518 value = 0.3064 dict = 1.00 all = 0.3162 } ---------------------------- total value len = 10.439040 GB avg = 0.194 KB (by entry num) total key len = 1.471884 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103602 Billion total entry num = 0.053911331 Billion write speed all = 0.576362067 MB/sec (with seq num) write speed all = 0.555492187 MB/sec (without seq num) 2024/11/23-14:39:37.963160 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263911: 129427 keys, 42443287 bytes, temperature: kWarm 2024/11/23-14:39:37.963228 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343977963194, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263911, "file_size": 42443287, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4031990438, "table_properties": {"data_size": 40326192, "index_size": 235840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630038, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343976, "oldest_key_time": 0, "file_creation_time": 1732343976, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263911, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:38.011666 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263912.sst) /dev/shm/ToplingTemp/Topling-77275-L8yNCn 2024/11/23-14:39:38.698615 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263912.sst): first pass time = 0.69's, 195.395'MB/sec 2024/11/23-14:39:39.478642 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263912.sst): old prealloc_size = 147639500, real_size = 40507248 2024/11/23-14:39:39.478774 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263912.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.01 2024/11/23-14:39:39.478854 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263912.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:39.638366 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263912.sst): second pass time = 0.78's, 168.921'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.854'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2182.372'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 988.897'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45907.537'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 168.921'MB/sec zip pipeline throughput = 168.921'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1016.99 avg-zval = 311.01 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1018.47 avg-zval = 311.01 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 27404 bytes = 102784 } seq expand size = 191828 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 7.0425 value = 3.2700 dict = 1.00 all = 3.1974 } Zip/UnZip{ index = 0.1420 value = 0.3058 dict = 1.00 all = 0.3128 } ---------------------------- total value len = 10.570862 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233033 Billion total entry num = 0.054040762 Billion write speed all = 0.582817422 MB/sec (with seq num) write speed all = 0.561899184 MB/sec (without seq num) 2024/11/23-14:39:39.690963 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263912: 129431 keys, 41977520 bytes, temperature: kWarm 2024/11/23-14:39:39.691044 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343979691008, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263912, "file_size": 41977520, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4031892846, "table_properties": {"data_size": 40254176, "index_size": 220544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629630, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343977, "oldest_key_time": 0, "file_creation_time": 1732343977, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263912, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:39.728080 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263913.sst) /dev/shm/ToplingTemp/Topling-77275-0CEN2k 2024/11/23-14:39:40.431240 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263913.sst): first pass time = 0.70's, 190.887'MB/sec 2024/11/23-14:39:41.276053 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263913.sst): old prealloc_size = 147639500, real_size = 41022784 2024/11/23-14:39:41.276173 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263913.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.01 avg-zval = 314.99 2024/11/23-14:39:41.276340 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263913.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:41.439055 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263913.sst): second pass time = 0.84's, 156.136'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.770'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2270.282'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 978.539'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46663.823'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 156.136'MB/sec zip pipeline throughput = 156.136'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.01 avg-zval = 314.99 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1023.17 avg-zval = 314.99 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 113883 bytes = 427072 } seq expand size = 797181 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9875 value = 3.2287 dict = 1.00 all = 3.1346 } Zip/UnZip{ index = 0.1431 value = 0.3097 dict = 1.00 all = 0.3190 } ---------------------------- total value len = 10.703288 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362461 Billion total entry num = 0.054170190 Billion write speed all = 0.589298821 MB/sec (with seq num) write speed all = 0.568332310 MB/sec (without seq num) 2024/11/23-14:39:41.489805 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263913: 129428 keys, 42818047 bytes, temperature: kWarm 2024/11/23-14:39:41.489874 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343981489838, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263913, "file_size": 42818047, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4031980847, "table_properties": {"data_size": 40767984, "index_size": 222272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629199, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343979, "oldest_key_time": 0, "file_creation_time": 1732343979, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263913, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:41.525112 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263914.sst) /dev/shm/ToplingTemp/Topling-77275-hRmSql 2024/11/23-14:39:42.106739 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263914.sst): first pass time = 0.58's, 230.784'MB/sec 2024/11/23-14:39:43.045337 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263914.sst): old prealloc_size = 147639500, real_size = 40884720 2024/11/23-14:39:43.045453 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263914.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1017.03 avg-zval = 313.92 2024/11/23-14:39:43.045592 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263914.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:43.209028 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263914.sst): second pass time = 0.94's, 140.500'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.310'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2327.467'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.04's, 974.743'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48986.743'MB/sec zip my value time = 0.94's, unzip length = 0.132'GB zip my value throughput = 140.500'MB/sec zip pipeline throughput = 143.806'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.03 avg-zval = 313.92 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1022.05 avg-zval = 313.92 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 92840 bytes = 348160 } seq expand size = 649880 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.9654 value = 3.2398 dict = 1.00 all = 3.1506 } Zip/UnZip{ index = 0.1436 value = 0.3087 dict = 1.00 all = 0.3174 } ---------------------------- total value len = 10.835568 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021175 GB avg = 0.015 KB total ukey num = 0.067491887 Billion total entry num = 0.054299616 Billion write speed all = 0.595772886 MB/sec (with seq num) write speed all = 0.574758081 MB/sec (without seq num) 2024/11/23-14:39:43.260558 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263914: 129426 keys, 42600882 bytes, temperature: kWarm 2024/11/23-14:39:43.260626 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343983260593, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263914, "file_size": 42600882, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4031908767, "table_properties": {"data_size": 40629216, "index_size": 222976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131630016, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343981, "oldest_key_time": 0, "file_creation_time": 1732343981, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263914, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:43.307649 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263915.sst) /dev/shm/ToplingTemp/Topling-77275-BbUvgl 2024/11/23-14:39:43.882312 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263915.sst): first pass time = 0.57's, 233.579'MB/sec 2024/11/23-14:39:44.724703 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263915.sst): old prealloc_size = 147639500, real_size = 40782384 2024/11/23-14:39:44.724831 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263915.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.99 avg-zval = 313.07 2024/11/23-14:39:44.724971 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263915.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:44.888696 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263915.sst): second pass time = 0.84's, 156.855'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.424'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2168.680'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.04's, 964.034'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 51293.671'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 156.855'MB/sec zip pipeline throughput = 156.855'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1016.99 avg-zval = 313.07 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1022.60 avg-zval = 313.07 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 103705 bytes = 388912 } seq expand size = 725935 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.7845 value = 3.2484 dict = 1.00 all = 3.1552 } Zip/UnZip{ index = 0.1474 value = 0.3078 dict = 1.00 all = 0.3169 } ---------------------------- total value len = 10.967924 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621318 Billion total entry num = 0.054429047 Billion write speed all = 0.602252163 MB/sec (with seq num) write speed all = 0.581188977 MB/sec (without seq num) 2024/11/23-14:39:44.940561 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263915: 129431 keys, 42538899 bytes, temperature: kWarm 2024/11/23-14:39:44.940630 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343984940595, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263915, "file_size": 42538899, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4031974252, "table_properties": {"data_size": 40520928, "index_size": 228928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131630121, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343983, "oldest_key_time": 0, "file_creation_time": 1732343983, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263915, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:44.979807 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263916.sst) /dev/shm/ToplingTemp/Topling-77275-2OJxOl 2024/11/23-14:39:45.526210 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263916.sst): first pass time = 0.55's, 245.656'MB/sec 2024/11/23-14:39:46.474569 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263916.sst): old prealloc_size = 147639500, real_size = 40453984 2024/11/23-14:39:46.474709 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263916.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.00 avg-zval = 310.59 2024/11/23-14:39:46.474790 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263916.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:46.635291 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263916.sst): second pass time = 0.95's, 138.906'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.862'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1883.119'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 986.797'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47407.493'MB/sec zip my value time = 0.95's, unzip length = 0.132'GB zip my value throughput = 138.906'MB/sec zip pipeline throughput = 199.661'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 310.59 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.43 avg-zval = 310.59 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 26401 bytes = 99024 } seq expand size = 184807 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.9936 value = 3.2744 dict = 1.00 all = 3.2017 } Zip/UnZip{ index = 0.1430 value = 0.3054 dict = 1.00 all = 0.3123 } ---------------------------- total value len = 11.099738 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750747 Billion total entry num = 0.054558476 Billion write speed all = 0.608702202 MB/sec (with seq num) write speed all = 0.587590713 MB/sec (without seq num) 2024/11/23-14:39:46.689551 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263916: 129429 keys, 41920696 bytes, temperature: kWarm 2024/11/23-14:39:46.689618 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343986689583, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263916, "file_size": 41920696, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4031967869, "table_properties": {"data_size": 40199376, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629342, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343984, "oldest_key_time": 0, "file_creation_time": 1732343984, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263916, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:46.725416 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263917.sst) /dev/shm/ToplingTemp/Topling-77275-FIUO3m 2024/11/23-14:39:47.443213 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263917.sst): first pass time = 0.72's, 186.996'MB/sec 2024/11/23-14:39:48.222043 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263917.sst): old prealloc_size = 147639500, real_size = 40585600 2024/11/23-14:39:48.222165 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263917.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.03 avg-zval = 311.61 2024/11/23-14:39:48.222252 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263917.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:48.383949 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263917.sst): second pass time = 0.78's, 169.315'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.757'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2211.468'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.271'MB/sec write SST data time = 0.04's, 995.072'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44085.600'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 169.315'MB/sec zip pipeline throughput = 169.315'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.03 avg-zval = 311.61 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.44 avg-zval = 311.61 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 26082 bytes = 97824 } seq expand size = 182574 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9935 value = 3.2637 dict = 1.00 all = 3.1918 } Zip/UnZip{ index = 0.1430 value = 0.3064 dict = 1.00 all = 0.3133 } ---------------------------- total value len = 11.231551 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880173 Billion total entry num = 0.054687902 Billion write speed all = 0.615151006 MB/sec (with seq num) write speed all = 0.593991225 MB/sec (without seq num) 2024/11/23-14:39:48.437769 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263917: 129426 keys, 42050826 bytes, temperature: kWarm 2024/11/23-14:39:48.437840 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343988437805, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263917, "file_size": 42050826, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4031977331, "table_properties": {"data_size": 40330992, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629878, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343986, "oldest_key_time": 0, "file_creation_time": 1732343986, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263917, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:48.475479 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263918.sst) /dev/shm/ToplingTemp/Topling-77275-vjDKNo 2024/11/23-14:39:48.973305 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263918.sst): first pass time = 0.50's, 269.631'MB/sec 2024/11/23-14:39:49.766952 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263918.sst): old prealloc_size = 147639500, real_size = 40451136 2024/11/23-14:39:49.767123 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263918.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.56 2024/11/23-14:39:49.767240 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263918.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:50.018318 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263918.sst): second pass time = 0.79's, 166.049'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 166.769'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1688.539'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.173'MB/sec write SST data time = 0.06's, 657.895'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47406.942'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 166.049'MB/sec zip pipeline throughput = 166.049'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 310.56 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.37 avg-zval = 310.56 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 25476 bytes = 95552 } seq expand size = 178332 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.9938 value = 3.2747 dict = 1.00 all = 3.2022 } Zip/UnZip{ index = 0.1430 value = 0.3054 dict = 1.00 all = 0.3123 } ---------------------------- total value len = 11.363359 GB avg = 0.207 KB (by entry num) total key len = 1.490004 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009604 Billion total entry num = 0.054817333 Billion write speed all = 0.621601964 MB/sec (with seq num) write speed all = 0.600393780 MB/sec (without seq num) 2024/11/23-14:39:50.073025 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263918: 129431 keys, 41914408 bytes, temperature: kWarm 2024/11/23-14:39:50.073093 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343990073060, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263918, "file_size": 41914408, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4031965354, "table_properties": {"data_size": 40196528, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131630061, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343988, "oldest_key_time": 0, "file_creation_time": 1732343988, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263918, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:50.110862 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263919.sst) /dev/shm/ToplingTemp/Topling-77275-hLXfmn 2024/11/23-14:39:50.676883 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263919.sst): first pass time = 0.57's, 237.143'MB/sec 2024/11/23-14:39:51.462643 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263919.sst): old prealloc_size = 147639500, real_size = 40749792 2024/11/23-14:39:51.462777 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263919.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.00 avg-zval = 312.88 2024/11/23-14:39:51.462915 139957559264832 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263919.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:51.627528 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263919.sst): second pass time = 0.78's, 167.875'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 257.413'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2249.628'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.265'MB/sec write SST data time = 0.04's, 995.975'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 40875.341'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 167.875'MB/sec zip pipeline throughput = 167.875'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 312.88 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1021.67 avg-zval = 312.88 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 86287 bytes = 323584 } seq expand size = 604009 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.9916 value = 3.2505 dict = 1.00 all = 3.1625 } Zip/UnZip{ index = 0.1430 value = 0.3076 dict = 1.00 all = 0.3162 } ---------------------------- total value len = 11.495592 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139033 Billion total entry num = 0.054946762 Billion write speed all = 0.628073221 MB/sec (with seq num) write speed all = 0.606816617 MB/sec (without seq num) 2024/11/23-14:39:51.680902 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263919: 129429 keys, 42440865 bytes, temperature: kWarm 2024/11/23-14:39:51.680963 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343991680931, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263919, "file_size": 42440865, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4031990665, "table_properties": {"data_size": 40495120, "index_size": 222144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629520, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343990, "oldest_key_time": 0, "file_creation_time": 1732343990, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263919, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:51.718140 139957559264832 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263920.sst) /dev/shm/ToplingTemp/Topling-77275-SJWqln 2024/11/23-14:39:51.729230 139957559264832 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263920.sst): first pass time = 0.01's, 419.401'MB/sec 2024/11/23-14:39:51.767503 139957559264832 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263920.sst): old prealloc_size = 147639500, real_size = 1490016 2024/11/23-14:39:51.767558 139957559264832 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263920.sst): index type = FixedLenKeyIndex , store type = DictZipBlobStore usrkeys = 4464 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 8.23 raw-val = 0.0045 GB zip-val = 0.0015 GB avg-val =1016.97 avg-zval = 325.27 2024/11/23-14:39:51.775597 139957559264832 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00/263920.sst): second pass time = 0.04's, 120.225'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.01's, 192.276'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 947.806'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.177'MB/sec write SST data time = 0.00's, 886.272'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.047'MB, throughput = 1710.269'MB/sec zip my value time = 0.04's, unzip length = 0.005'GB zip my value throughput = 120.225'MB/sec zip pipeline throughput = 120.225'MB/sec entries = 4464 avg-key = 20.00 avg-zkey = 8.23 avg-val = 1016.97 avg-zval = 325.27 usrkeys = 4464 avg-key = 12.00 avg-zkey = 8.23 avg-val = 1023.97 avg-zval = 325.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 4464 bytes = 16752 } seq expand size = 31248 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0001 GB value = 0.0045 GB dict = 0.05 MB all = 0.0046 GB } __ZipSize{ index = 0.0000 GB value = 0.0015 GB dict = 0.05 MB all = 0.0016 GB } UnZip/Zip{ index = 1.4582 value = 3.1266 dict = 1.00 all = 2.9796 } Zip/UnZip{ index = 0.6858 value = 0.3198 dict = 1.00 all = 0.3356 } ---------------------------- total value len = 11.500163 GB avg = 0.209 KB (by entry num) total key len = 1.492682 GB avg = 0.027 KB total ukey len = 1.028995 GB avg = 0.015 KB total ukey num = 0.068143497 Billion total entry num = 0.054951226 Billion write speed all = 0.628294081 MB/sec (with seq num) write speed all = 0.607035902 MB/sec (without seq num) 2024/11/23-14:39:51.785684 139957559264832 [compaction_job.cc:1948] [default] [JOB 28109] Generated table #263920: 4464 keys, 1553569 bytes, temperature: kWarm 2024/11/23-14:39:51.785733 139957559264832 EVENT_LOG_v1 {"time_micros": 1732343991785706, "cf_name": "default", "job": 28109, "event": "table_file_creation", "file_number": 263920, "file_size": 1553569, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3354484105, "largest_seqno": 4030479583, "table_properties": {"data_size": 1451984, "index_size": 36736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 89280, "raw_average_key_size": 20, "raw_value_size": 4539750, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 4464, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-FixedLenKeyIndex-DictZipBlobStore", "creation_time": 1732343991, "oldest_key_time": 0, "file_creation_time": 1732343991, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263920, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:51.796659 139957559264832 [compaction_job.cc:703] job-28109: subcompact[0], size: 0.851140 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28109/att-00 [263900,263901,263902,263903,263904,263905,263906,263907,263908,263909,263910,263911,263912,263913,263914,263915,263916,263917,263918,263919,263920] 2024/11/23-14:39:51.940135 139957559264832 (Original Log Time 2024/11/23-14:39:12.358993) EVENT_LOG_v1 {"time_micros": 1732343952358945, "job": 28109, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [263896], "files_L3": [263851, 263852, 263853, 263854, 263855, 263856, 263857, 263858, 263859, 263860, 263861, 263862, 263863, 263864, 263865, 263866, 263867, 263868, 263869, 263872], "score": 0.998312, "input_data_size": 882285760, "oldest_snapshot_seqno": 4032018301} 2024/11/23-14:39:51.940138 139957559264832 (Original Log Time 2024/11/23-14:39:51.938410) [compaction_job.cc:2022] [default] [JOB 28109] Compacted 1@2 + 20@3 files to L3 => 851139537 bytes 2024/11/23-14:39:51.940140 139957559264832 (Original Log Time 2024/11/23-14:39:51.940064) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 22.4 rd, 21.6 wr, level 3, files in(1, 20) out(21 +0 blob) MB in(31.2, 810.2 +0.0 blob) out(811.7 +0.0 blob), read-write-amplify(53.0) write-amplify(26.0) OK, records in: 2620506, records dropped: 27444 output_compression: Snappy 2024/11/23-14:39:51.940143 139957559264832 (Original Log Time 2024/11/23-14:39:51.940108) EVENT_LOG_v1 {"time_micros": 1732343991940088, "cf": "default", "job": 28109, "event": "compaction_finished", "compaction_time_micros": 39437671, "compaction_time_cpu_micros": 8133233, "output_level": 3, "num_output_files": 21, "total_output_size": 851139537, "num_input_records": 2620506, "num_output_records": 2593062, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 625124172, "file_range_sync_nanos": 0, "file_fsync_nanos": 15844, "file_prepare_write_nanos": 49975, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/23-14:39:52.078935 139957559264832 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732343952,"executesMs":39716,"compactionJobId":28109,"attempt":0,"compactionInputRawBytes":2717455264,"compactionInputZipBytes":882285760,"compactionOutputRawBytes":0,"compactionOutputZipBytes":851139537,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-9e09db5599d6772f1c72a71771510fd8-932b8dacec2452a5-00"}}, response = 2024/11/23-14:39:52.079046 139957559264832 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28109/att-00: olev 3, work 39.717 s, result 9.784 ms, install 1.750 ms, input{raw 2.531 GiB zip 841.413 MiB}