2024/11/23-21:02:11.940678 139957483730496 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00: fork to child time = 0.007337 sec 2024/11/23-21:02:11.941389 139957483730496 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00 2024/11/23-21:02:11.950917 139957483730496 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00 2024/11/23-21:02:11.951807 139957483730496 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-21:02:11.952068 139957483730496 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-21:02:12.008791 139957483730496 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35741/att-00/MANIFEST-206347 2024/11/23-21:02:12.042590 139957483730496 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35741/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 280559, last_sequence is 4260463563, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 280433 2024/11/23-21:02:12.042596 139957483730496 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-21:02:12.042839 139957483730496 [version_set.cc:5721] Creating manifest 280559 2024/11/23-21:02:12.076475 139957483730496 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-21:02:12.257129 139957483730496 [compaction_job.cc:2437] [default] [JOB 35741] Compacting 21@3 + 54@4 files to L4, score -1.00, subcompactions 1 : 1 2024/11/23-21:02:12.257185 139957483730496 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 26757 Base level 3, inputs: [280463(10KB) 280457(13KB) 280506(124MB) 280507(124MB) 280508(124MB) 280509(978KB) 280467(17MB) 280459(314KB) 280461(5602KB) 280462(33MB) 262667(33MB) 262668(34MB) 262669(34MB) 262670(35MB) 262671(35MB) 264939(37MB) 280500(35MB) 280501(35MB) 280502(38MB) 280503(35MB) 280537(38MB)], [280425(3339B) 280426(23KB) 267485(249MB) 262338(249MB) 274572(201MB) 274573(151MB) 271322(196MB) 271323(180MB) 234406(102MB) 270325(97MB) 213920(53KB) 227563(126KB) 237538(759KB) 271321(785KB) 280231(1393KB) 280427(1394KB) 275661(16MB) 279539(149MB) 279540(58MB) 277299(147MB) 277300(31MB) 206487(38MB) 244184(72MB) 245367(70MB) 247941(66MB) 255717(64MB) 262472(64MB) 262473(50MB) 258439(63MB) 259954(66MB) 259955(34MB) 260685(61MB) 260686(31MB) 259249(63MB) 259250(42MB) 276084(69MB) 276085(29MB) 275122(65MB) 280015(64MB) 280016(20MB) 279794(65MB) 279795(29MB) 276655(66MB) 276656(56MB) 264775(35MB) 261882(34MB) 262673(34MB) 263783(34MB) 271154(11MB) 272081(4868KB) 266193(62MB) 266194(22MB) 267895(35MB) 269339(60MB)] 2024/11/23-21:02:12.261137 139957483730496 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:02:12.261168 139957483730496 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:02:12.261188 139957483730496 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:02:12.261198 139957483730496 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:02:12.261203 139957483730496 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:02:12.277055 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280560.sst) /dev/shm/ToplingTemp/Topling-188284-Is3E3n 2024/11/23-21:02:12.292854 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280560.sst): first pass time = 0.02's, 0.109'MB/sec 2024/11/23-21:02:12.313722 139957483730496 [top_zip_table_builder.cc:2726] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280560.sst): part-0: reject ZipStore ratio = 0.956 2024/11/23-21:02:12.314063 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280560.sst): old prealloc_size = 295279001, real_size = 2296 2024/11/23-21:02:12.314087 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280560.sst): index type = UintIndex_IL_256_32 , store type = ZipOffsetBlobStore usrkeys = 13 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 4.00 avg-zkey = 24.00 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 115.85 avg-zval = 136.62 2024/11/23-21:02:12.314565 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280560.sst): second pass time = 0.02's, 0.074'MB/sec, value only(87.9% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 6.800'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 13.450'MB/sec (index lex order gen) rebuild zvType time = 0.00's, 0.028'MB/sec write SST data time = 0.00's, 9.519'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.02's, unzip length = 0.000'GB zip my value throughput = 0.074'MB/sec zip pipeline throughput = 0.074'MB/sec entries = 13 avg-key = 16.00 avg-zkey = 24.62 avg-val = 115.85 avg-zval = 136.62 usrkeys = 13 avg-key = 8.00 avg-zkey = 24.62 avg-val = 122.85 avg-zval = 136.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 13 bytes = 64 } seq expand size = 91 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } __ZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } UnZip/Zip{ index = 0.3250 value = 0.8480 dict = 0.00 all = 0.5133 } Zip/UnZip{ index = 3.0769 value = 1.1793 dict = 0.00 all = 1.9481 } ---------------------------- total value len = 8.848621 GB avg = 0.169 KB (by entry num) total key len = 1.440821 GB avg = 0.028 KB total ukey len = 0.997878 GB avg = 0.015 KB total ukey num = 0.065550448 Billion total entry num = 0.052358177 Billion write speed all = 0.235887646 MB/sec (with seq num) write speed all = 0.226285068 MB/sec (without seq num) 2024/11/23-21:02:12.320180 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280560: 13 keys, 3339 bytes, temperature: kWarm 2024/11/23-21:02:12.320247 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366932320202, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280560, "file_size": 3339, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3236831185, "largest_seqno": 4259662192, "table_properties": {"data_size": 1776, "index_size": 320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 208, "raw_average_key_size": 16, "raw_value_size": 1506, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 13, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-ZipOffsetBlobStore", "creation_time": 1732366932, "oldest_key_time": 0, "file_creation_time": 1732366932, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280560, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:02:12.340690 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280561.sst) /dev/shm/ToplingTemp/Topling-188284-BC6W1k 2024/11/23-21:02:16.466106 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280561.sst): first pass time = 4.13's, 0.005'MB/sec 2024/11/23-21:02:16.487242 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280561.sst): old prealloc_size = 295279001, real_size = 22672 2024/11/23-21:02:16.487264 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280561.sst): index type = FixedLenKeyIndex , store type = DictZipBlobStore usrkeys = 139 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 66.76 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 120.73 avg-zval = 94.85 2024/11/23-21:02:16.487732 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280561.sst): second pass time = 0.02's, 0.816'MB/sec, value only(85.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 51.401'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 640.265'MB/sec (index lex order gen) rebuild zvType time = 0.00's, 0.148'MB/sec write SST data time = 0.00's, 107.354'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 9.011'MB/sec zip my value time = 0.02's, unzip length = 0.000'GB zip my value throughput = 0.816'MB/sec zip pipeline throughput = 0.816'MB/sec entries = 139 avg-key = 20.00 avg-zkey = 66.76 avg-val = 120.73 avg-zval = 94.85 usrkeys = 139 avg-key = 12.00 avg-zkey = 66.76 avg-val = 127.73 avg-zval = 94.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 139 bytes = 544 } seq expand size = 973 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } __ZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } UnZip/Zip{ index = 0.1797 value = 1.2729 dict = 1.00 all = 0.8011 } Zip/UnZip{ index = 5.5635 value = 0.7856 dict = 1.00 all = 1.2483 } ---------------------------- total value len = 8.848639 GB avg = 0.169 KB (by entry num) total key len = 1.440824 GB avg = 0.028 KB total ukey len = 0.997880 GB avg = 0.015 KB total ukey num = 0.065550587 Billion total entry num = 0.052358316 Billion write speed all = 0.235865551 MB/sec (with seq num) write speed all = 0.226263866 MB/sec (without seq num) 2024/11/23-21:02:16.493430 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280561: 139 keys, 24420 bytes, temperature: kWarm 2024/11/23-21:02:16.493475 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366936493449, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280561, "file_size": 24420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3236115370, "largest_seqno": 4259662191, "table_properties": {"data_size": 13184, "index_size": 9280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2780, "raw_average_key_size": 20, "raw_value_size": 16782, "raw_average_value_size": 120, "num_data_blocks": 1, "num_entries": 139, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-FixedLenKeyIndex-DictZipBlobStore", "creation_time": 1732366932, "oldest_key_time": 0, "file_creation_time": 1732366932, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280561, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:02:16.518063 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280562.sst) /dev/shm/ToplingTemp/Topling-188284-skDARn 2024/11/23-21:02:16.853406 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280562.sst): first pass time = 0.34's, 800.548'MB/sec 2024/11/23-21:02:20.817292 139957483730496 [top_zip_table_builder.cc:2726] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280562.sst): part-0: reject ZipStore ratio = 0.805 2024/11/23-21:02:21.342024 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280562.sst): old prealloc_size = 295279001, real_size = 260098640 2024/11/23-21:02:21.342275 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280562.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 397428 prefix = 4 raw-key = 0.0048 GB zip-key = 0.0007 GB avg-key = 12.00 avg-zkey = 1.71 raw-val = 0.2589 GB zip-val = 0.2593 GB avg-val = 651.43 avg-zval = 652.49 2024/11/23-21:02:21.342672 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280562.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:02:21.949370 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280562.sst): second pass time = 3.96's, 65.337'MB/sec, value only(96.4% of KV) wait indexing time = 0.53's, remap KeyValue time = 0.61's, 430.545'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2964.048'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.201'MB/sec write SST data time = 0.11's, 2318.495'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 3.96's, unzip length = 0.259'GB zip my value throughput = 65.337'MB/sec zip pipeline throughput = 65.337'MB/sec entries = 397428 avg-key = 24.00 avg-zkey = 1.71 avg-val = 651.43 avg-zval = 652.49 usrkeys = 397428 avg-key = 16.00 avg-zkey = 1.71 avg-val = 658.08 avg-zval = 652.49 TagRS{ kinds = RS_Key0_TagN bytes = 62200 } TagArray{ size = 377597 bytes = 1416000 } seq expand size = 2643179 multi value expand size = 0 cnt WriteAppend = 31614 UnZipSize{ index = 0.0064 GB value = 0.2589 GB dict = 0.00 MB all = 0.2684 GB } __ZipSize{ index = 0.0007 GB value = 0.2593 GB dict = 0.00 MB all = 0.2615 GB } UnZip/Zip{ index = 9.3556 value = 0.9984 dict = 0.00 all = 1.0266 } Zip/UnZip{ index = 0.1069 value = 1.0016 dict = 0.00 all = 0.9741 } ---------------------------- total value len = 9.110180 GB avg = 0.173 KB (by entry num) total key len = 1.450362 GB avg = 0.027 KB total ukey len = 1.004239 GB avg = 0.015 KB total ukey num = 0.065948015 Billion total entry num = 0.052755744 Billion write speed all = 0.242049190 MB/sec (with seq num) write speed all = 0.232375834 MB/sec (without seq num) 2024/11/23-21:02:22.224649 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280562: 397428 keys, 261481823 bytes, temperature: kWarm 2024/11/23-21:02:22.224718 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366942224683, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280562, "file_size": 261481823, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4259661721, "table_properties": {"data_size": 259319424, "index_size": 679680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9538272, "raw_average_key_size": 24, "raw_value_size": 258897285, "raw_average_value_size": 651, "num_data_blocks": 1, "num_entries": 397428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732366936, "oldest_key_time": 0, "file_creation_time": 1732366936, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280562, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:02:22.321381 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280563.sst) /dev/shm/ToplingTemp/Topling-188284-6H4Unm 2024/11/23-21:02:27.033583 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280563.sst): first pass time = 4.71's, 56.966'MB/sec 2024/11/23-21:02:30.627871 139957483730496 [top_zip_table_builder.cc:2726] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280563.sst): part-0: reject ZipStore ratio = 0.803 2024/11/23-21:02:31.152429 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280563.sst): old prealloc_size = 295279001, real_size = 260077632 2024/11/23-21:02:31.152663 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280563.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 398316 prefix = 4 raw-key = 0.0048 GB zip-key = 0.0007 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.2589 GB zip-val = 0.2593 GB avg-val = 649.93 avg-zval = 650.99 2024/11/23-21:02:31.153014 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280563.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:02:31.759466 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280563.sst): second pass time = 3.59's, 72.049'MB/sec, value only(96.4% of KV) wait indexing time = 0.53's, remap KeyValue time = 0.61's, 430.227'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3145.630'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.203'MB/sec write SST data time = 0.12's, 2259.303'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 3.59's, unzip length = 0.259'GB zip my value throughput = 72.049'MB/sec zip pipeline throughput = 72.049'MB/sec entries = 398316 avg-key = 24.00 avg-zkey = 1.70 avg-val = 649.93 avg-zval = 650.99 usrkeys = 398316 avg-key = 16.00 avg-zkey = 1.70 avg-val = 655.11 avg-zval = 650.99 TagRS{ kinds = RS_Key0_TagN bytes = 62320 } TagArray{ size = 295061 bytes = 1106496 } seq expand size = 2065427 multi value expand size = 0 cnt WriteAppend = 31611 UnZipSize{ index = 0.0064 GB value = 0.2589 GB dict = 0.00 MB all = 0.2684 GB } __ZipSize{ index = 0.0007 GB value = 0.2593 GB dict = 0.00 MB all = 0.2612 GB } UnZip/Zip{ index = 9.3898 value = 0.9984 dict = 0.00 all = 1.0279 } Zip/UnZip{ index = 0.1065 value = 1.0016 dict = 0.00 all = 0.9729 } ---------------------------- total value len = 9.371121 GB avg = 0.176 KB (by entry num) total key len = 1.459922 GB avg = 0.027 KB total ukey len = 1.010612 GB avg = 0.015 KB total ukey num = 0.066346331 Billion total entry num = 0.053154060 Billion write speed all = 0.248193308 MB/sec (with seq num) write speed all = 0.238449107 MB/sec (without seq num) 2024/11/23-21:02:32.044528 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280563: 398316 keys, 261151135 bytes, temperature: kWarm 2024/11/23-21:02:32.044594 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366952044560, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280563, "file_size": 261151135, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4259660659, "table_properties": {"data_size": 259299120, "index_size": 678720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9559584, "raw_average_key_size": 24, "raw_value_size": 258876009, "raw_average_value_size": 649, "num_data_blocks": 1, "num_entries": 398316, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732366942, "oldest_key_time": 0, "file_creation_time": 1732366942, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280563, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:02:32.136279 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280564.sst) /dev/shm/ToplingTemp/Topling-188284-tEUsgn 2024/11/23-21:02:34.945458 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280564.sst): first pass time = 2.81's, 95.558'MB/sec 2024/11/23-21:02:38.944258 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280564.sst): old prealloc_size = 295279001, real_size = 207848432 2024/11/23-21:02:38.944516 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280564.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 397483 prefix = 4 raw-key = 0.0048 GB zip-key = 0.0007 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.2589 GB zip-val = 0.2071 GB avg-val = 651.34 avg-zval = 520.96 2024/11/23-21:02:38.944870 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280564.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:02:39.500248 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280564.sst): second pass time = 4.00's, 64.764'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.56's, 380.869'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3056.052'MB/sec (index lex order gen) rebuild zvType time = 0.43's, 0.231'MB/sec write SST data time = 0.13's, 1675.909'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.630'MB, throughput = 81387.869'MB/sec zip my value time = 4.00's, unzip length = 0.259'GB zip my value throughput = 64.764'MB/sec zip pipeline throughput = 64.764'MB/sec entries = 397483 avg-key = 24.00 avg-zkey = 1.70 avg-val = 651.34 avg-zval = 520.96 usrkeys = 397483 avg-key = 16.00 avg-zkey = 1.70 avg-val = 657.47 avg-zval = 520.96 TagRS{ kinds = RS_Key0_TagN bytes = 62200 } TagArray{ size = 347895 bytes = 1304624 } seq expand size = 2435265 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0064 GB value = 0.2589 GB dict = 2.63 MB all = 0.2684 GB } __ZipSize{ index = 0.0007 GB value = 0.2071 GB dict = 2.63 MB all = 0.2118 GB } UnZip/Zip{ index = 9.3941 value = 1.2503 dict = 1.00 all = 1.2677 } Zip/UnZip{ index = 0.1064 value = 0.7998 dict = 1.00 all = 0.7888 } ---------------------------- total value len = 9.632453 GB avg = 0.180 KB (by entry num) total key len = 1.469462 GB avg = 0.027 KB total ukey len = 1.016971 GB avg = 0.015 KB total ukey num = 0.066743814 Billion total entry num = 0.053551543 Billion write speed all = 0.254355208 MB/sec (with seq num) write speed all = 0.244539881 MB/sec (without seq num) 2024/11/23-21:02:39.730141 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280564: 397483 keys, 211750313 bytes, temperature: kWarm 2024/11/23-21:02:39.730207 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366959730173, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280564, "file_size": 211750313, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4259662193, "table_properties": {"data_size": 207071904, "index_size": 676992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9539592, "raw_average_key_size": 24, "raw_value_size": 258896563, "raw_average_value_size": 651, "num_data_blocks": 1, "num_entries": 397483, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366952, "oldest_key_time": 0, "file_creation_time": 1732366952, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280564, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:02:39.798663 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280565.sst) /dev/shm/ToplingTemp/Topling-188284-6qxqHj 2024/11/23-21:02:41.714635 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280565.sst): first pass time = 1.92's, 140.107'MB/sec 2024/11/23-21:02:45.492845 139957483730496 [top_zip_table_builder.cc:2726] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280565.sst): part-0: reject ZipStore ratio = 0.803 2024/11/23-21:02:46.019799 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280565.sst): old prealloc_size = 295279001, real_size = 260076480 2024/11/23-21:02:46.020058 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280565.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 398378 prefix = 4 raw-key = 0.0048 GB zip-key = 0.0007 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.2589 GB zip-val = 0.2593 GB avg-val = 649.82 avg-zval = 650.88 2024/11/23-21:02:46.020388 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280565.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:02:46.638331 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280565.sst): second pass time = 3.78's, 68.539'MB/sec, value only(96.4% of KV) wait indexing time = 0.53's, remap KeyValue time = 0.62's, 422.163'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2829.322'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.196'MB/sec write SST data time = 0.11's, 2351.096'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 3.78's, unzip length = 0.259'GB zip my value throughput = 68.539'MB/sec zip pipeline throughput = 68.539'MB/sec entries = 398378 avg-key = 24.00 avg-zkey = 1.70 avg-val = 649.82 avg-zval = 650.88 usrkeys = 398378 avg-key = 16.00 avg-zkey = 1.70 avg-val = 654.84 avg-zval = 650.88 TagRS{ kinds = RS_Key0_TagN bytes = 62360 } TagArray{ size = 285811 bytes = 1071808 } seq expand size = 2000677 multi value expand size = 0 cnt WriteAppend = 31611 UnZipSize{ index = 0.0064 GB value = 0.2589 GB dict = 0.00 MB all = 0.2684 GB } __ZipSize{ index = 0.0007 GB value = 0.2593 GB dict = 0.00 MB all = 0.2611 GB } UnZip/Zip{ index = 9.3939 value = 0.9984 dict = 0.00 all = 1.0280 } Zip/UnZip{ index = 0.1065 value = 1.0016 dict = 0.00 all = 0.9727 } ---------------------------- total value len = 9.893329 GB avg = 0.183 KB (by entry num) total key len = 1.479023 GB avg = 0.027 KB total ukey len = 1.023345 GB avg = 0.015 KB total ukey num = 0.067142192 Billion total entry num = 0.053949921 Billion write speed all = 0.260508562 MB/sec (with seq num) write speed all = 0.250621835 MB/sec (without seq num) 2024/11/23-21:02:46.914602 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280565: 398378 keys, 261115359 bytes, temperature: kWarm 2024/11/23-21:02:46.914671 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366966914636, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280565, "file_size": 261115359, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4259660536, "table_properties": {"data_size": 259298160, "index_size": 678528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9561072, "raw_average_key_size": 24, "raw_value_size": 258874993, "raw_average_value_size": 649, "num_data_blocks": 1, "num_entries": 398378, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732366959, "oldest_key_time": 0, "file_creation_time": 1732366959, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280565, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:02:47.022496 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280566.sst) /dev/shm/ToplingTemp/Topling-188284-n4vzCl 2024/11/23-21:02:51.238445 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280566.sst): first pass time = 4.22's, 63.672'MB/sec 2024/11/23-21:02:54.968643 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280566.sst): old prealloc_size = 295279001, real_size = 202707488 2024/11/23-21:02:54.968916 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280566.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 397635 prefix = 4 raw-key = 0.0048 GB zip-key = 0.0007 GB avg-key = 12.00 avg-zkey = 1.71 raw-val = 0.2589 GB zip-val = 0.2019 GB avg-val = 651.08 avg-zval = 507.82 2024/11/23-21:02:54.969347 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280566.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:02:55.522637 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280566.sst): second pass time = 3.73's, 69.430'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.55's, 373.266'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2894.783'MB/sec (index lex order gen) rebuild zvType time = 0.43's, 0.232'MB/sec write SST data time = 0.12's, 1662.654'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.627'MB, throughput = 67188.255'MB/sec zip my value time = 3.73's, unzip length = 0.259'GB zip my value throughput = 69.430'MB/sec zip pipeline throughput = 69.430'MB/sec entries = 397635 avg-key = 24.00 avg-zkey = 1.71 avg-val = 651.08 avg-zval = 507.82 usrkeys = 397635 avg-key = 16.00 avg-zkey = 1.71 avg-val = 658.08 avg-zval = 507.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 397635 bytes = 1540848 } seq expand size = 2783445 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0064 GB value = 0.2589 GB dict = 2.63 MB all = 0.2684 GB } __ZipSize{ index = 0.0007 GB value = 0.2019 GB dict = 2.63 MB all = 0.2068 GB } UnZip/Zip{ index = 9.3667 value = 1.2821 dict = 1.00 all = 1.2982 } Zip/UnZip{ index = 0.1068 value = 0.7800 dict = 1.00 all = 0.7703 } ---------------------------- total value len = 10.155004 GB avg = 0.187 KB (by entry num) total key len = 1.488566 GB avg = 0.027 KB total ukey len = 1.029708 GB avg = 0.015 KB total ukey num = 0.067539827 Billion total entry num = 0.054347556 Billion write speed all = 0.266667156 MB/sec (with seq num) write speed all = 0.256709586 MB/sec (without seq num) 2024/11/23-21:02:55.745533 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280566: 397635 keys, 206780016 bytes, temperature: kWarm 2024/11/23-21:02:55.745602 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366975745567, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280566, "file_size": 206780016, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3183848655, "largest_seqno": 4259659867, "table_properties": {"data_size": 201928656, "index_size": 679232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9543240, "raw_average_key_size": 24, "raw_value_size": 258892331, "raw_average_value_size": 651, "num_data_blocks": 1, "num_entries": 397635, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366967, "oldest_key_time": 0, "file_creation_time": 1732366967, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280566, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:02:55.814265 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280567.sst) /dev/shm/ToplingTemp/Topling-188284-oa8oYl 2024/11/23-21:02:58.375172 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280567.sst): first pass time = 2.56's, 104.822'MB/sec 2024/11/23-21:03:01.926203 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280567.sst): old prealloc_size = 295279001, real_size = 206943840 2024/11/23-21:03:01.926469 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280567.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 397879 prefix = 4 raw-key = 0.0048 GB zip-key = 0.0007 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.2589 GB zip-val = 0.2062 GB avg-val = 650.67 avg-zval = 518.17 2024/11/23-21:03:01.926820 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280567.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:03:02.485549 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280567.sst): second pass time = 3.55's, 72.931'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.56's, 376.723'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3096.377'MB/sec (index lex order gen) rebuild zvType time = 0.44's, 0.228'MB/sec write SST data time = 0.12's, 1726.829'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.627'MB, throughput = 82808.342'MB/sec zip my value time = 3.55's, unzip length = 0.259'GB zip my value throughput = 72.931'MB/sec zip pipeline throughput = 72.931'MB/sec entries = 397879 avg-key = 24.00 avg-zkey = 1.70 avg-val = 650.67 avg-zval = 518.17 usrkeys = 397879 avg-key = 16.00 avg-zkey = 1.70 avg-val = 656.16 avg-zval = 518.17 TagRS{ kinds = RS_Key0_TagN bytes = 62280 } TagArray{ size = 312361 bytes = 1171376 } seq expand size = 2186527 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0064 GB value = 0.2589 GB dict = 2.63 MB all = 0.2684 GB } __ZipSize{ index = 0.0007 GB value = 0.2062 GB dict = 2.63 MB all = 0.2107 GB } UnZip/Zip{ index = 9.4061 value = 1.2557 dict = 1.00 all = 1.2740 } Zip/UnZip{ index = 0.1063 value = 0.7964 dict = 1.00 all = 0.7850 } ---------------------------- total value len = 10.416078 GB avg = 0.190 KB (by entry num) total key len = 1.498115 GB avg = 0.027 KB total ukey len = 1.036074 GB avg = 0.015 KB total ukey num = 0.067937706 Billion total entry num = 0.054745435 Billion write speed all = 0.272821587 MB/sec (with seq num) write speed all = 0.262792716 MB/sec (without seq num) 2024/11/23-21:03:02.718415 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280567: 397879 keys, 210709403 bytes, temperature: kWarm 2024/11/23-21:03:02.718486 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366982718450, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280567, "file_size": 210709403, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4259660878, "table_properties": {"data_size": 206167376, "index_size": 676800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9549096, "raw_average_key_size": 24, "raw_value_size": 258886820, "raw_average_value_size": 650, "num_data_blocks": 1, "num_entries": 397879, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366975, "oldest_key_time": 0, "file_creation_time": 1732366975, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280567, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:03:02.787809 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280568.sst) /dev/shm/ToplingTemp/Topling-188284-Ljkgwn 2024/11/23-21:03:05.923312 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280568.sst): first pass time = 3.14's, 77.352'MB/sec 2024/11/23-21:03:09.304029 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280568.sst): old prealloc_size = 295279001, real_size = 187239184 2024/11/23-21:03:09.304274 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280568.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 360204 prefix = 4 raw-key = 0.0043 GB zip-key = 0.0006 GB avg-key = 12.00 avg-zkey = 1.69 raw-val = 0.2339 GB zip-val = 0.1865 GB avg-val = 649.32 avg-zval = 517.87 2024/11/23-21:03:09.304536 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280568.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.243 GB 2024/11/23-21:03:09.812959 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280568.sst): second pass time = 3.38's, 69.206'MB/sec, value only(96.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.51's, 374.160'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3091.722'MB/sec (index lex order gen) rebuild zvType time = 0.39's, 0.228'MB/sec write SST data time = 0.11's, 1673.970'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.366'MB, throughput = 72844.011'MB/sec zip my value time = 3.38's, unzip length = 0.234'GB zip my value throughput = 69.206'MB/sec zip pipeline throughput = 69.206'MB/sec entries = 360204 avg-key = 24.00 avg-zkey = 1.69 avg-val = 649.32 avg-zval = 517.87 usrkeys = 360204 avg-key = 16.00 avg-zkey = 1.69 avg-val = 653.66 avg-zval = 517.87 TagRS{ kinds = RS_Key0_TagN bytes = 56400 } TagArray{ size = 223298 bytes = 837376 } seq expand size = 1563086 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0058 GB value = 0.2339 GB dict = 2.37 MB all = 0.2425 GB } __ZipSize{ index = 0.0006 GB value = 0.1865 GB dict = 2.37 MB all = 0.1904 GB } UnZip/Zip{ index = 9.4552 value = 1.2538 dict = 1.00 all = 1.2737 } Zip/UnZip{ index = 0.1058 value = 0.7976 dict = 1.00 all = 0.7851 } ---------------------------- total value len = 10.651530 GB avg = 0.193 KB (by entry num) total key len = 1.506760 GB avg = 0.027 KB total ukey len = 1.041837 GB avg = 0.015 KB total ukey num = 0.068297910 Billion total entry num = 0.055105639 Billion write speed all = 0.278364426 MB/sec (with seq num) write speed all = 0.268271264 MB/sec (without seq num) 2024/11/23-21:03:10.025856 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280568: 360204 keys, 190413176 bytes, temperature: kWarm 2024/11/23-21:03:10.025931 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366990025893, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280568, "file_size": 190413176, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4259662040, "table_properties": {"data_size": 186539392, "index_size": 609536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 8644896, "raw_average_key_size": 24, "raw_value_size": 233889032, "raw_average_value_size": 649, "num_data_blocks": 1, "num_entries": 360204, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732366982, "oldest_key_time": 0, "file_creation_time": 1732366982, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280568, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:03:10.095760 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280569.sst) /dev/shm/ToplingTemp/Topling-188284-6ZXkLl 2024/11/23-21:03:10.478641 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280569.sst): first pass time = 0.38's, 41.584'MB/sec 2024/11/23-21:03:10.695213 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280569.sst): old prealloc_size = 295279001, real_size = 5509040 2024/11/23-21:03:10.695264 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280569.sst): index type = FixedLenKeyIndex , store type = DictZipBlobStore usrkeys = 222537 prefix = 4 raw-key = 0.0018 GB zip-key = 0.0009 GB avg-key = 8.00 avg-zkey = 4.12 raw-val = 0.0115 GB zip-val = 0.0045 GB avg-val = 51.54 avg-zval = 20.39 2024/11/23-21:03:10.871376 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280569.sst): second pass time = 0.22's, 53.133'MB/sec, value only(72.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 31.859'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 33769.781'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.354'MB/sec write SST data time = 0.02's, 298.791'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.118'MB, throughput = 2993.719'MB/sec zip my value time = 0.22's, unzip length = 0.011'GB zip my value throughput = 53.133'MB/sec zip pipeline throughput = 53.133'MB/sec entries = 222537 avg-key = 20.00 avg-zkey = 4.12 avg-val = 51.54 avg-zval = 20.39 usrkeys = 222537 avg-key = 12.00 avg-zkey = 4.12 avg-val = 51.59 avg-zval = 20.39 TagRS{ kinds = RS_Key0_TagN bytes = 34880 } TagArray{ size = 1529 bytes = 3264 } seq expand size = 10703 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0027 GB value = 0.0115 GB dict = 0.12 MB all = 0.0159 GB } __ZipSize{ index = 0.0009 GB value = 0.0045 GB dict = 0.12 MB all = 0.0056 GB } UnZip/Zip{ index = 2.9156 value = 2.5277 dict = 1.00 all = 2.8369 } Zip/UnZip{ index = 0.3430 value = 0.3956 dict = 1.00 all = 0.3525 } ---------------------------- total value len = 10.663010 GB avg = 0.193 KB (by entry num) total key len = 1.511211 GB avg = 0.027 KB total ukey len = 1.044507 GB avg = 0.015 KB total ukey num = 0.068520447 Billion total entry num = 0.055328176 Billion write speed all = 0.278722402 MB/sec (with seq num) write speed all = 0.268588725 MB/sec (without seq num) 2024/11/23-21:03:10.884655 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280569: 222537 keys, 5611629 bytes, temperature: kWarm 2024/11/23-21:03:10.884714 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366990884682, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280569, "file_size": 5611629, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3369262432, "table_properties": {"data_size": 4537312, "index_size": 915904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4450740, "raw_average_key_size": 20, "raw_value_size": 11469071, "raw_average_value_size": 51, "num_data_blocks": 1, "num_entries": 222537, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-FixedLenKeyIndex-DictZipBlobStore", "creation_time": 1732366990, "oldest_key_time": 0, "file_creation_time": 1732366990, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280569, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:03:10.906010 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280570.sst) /dev/shm/ToplingTemp/Topling-188284-1Q968l 2024/11/23-21:03:14.827821 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280570.sst): first pass time = 3.92's, 21.162'MB/sec 2024/11/23-21:03:15.080371 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280570.sst): old prealloc_size = 295279001, real_size = 6670944 2024/11/23-21:03:15.081736 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280570.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZeroLengthBlobStore usrkeys = 3458056 prefix = 4 raw-key = 0.0415 GB zip-key = 0.0058 GB avg-key = 12.00 avg-zkey = 1.68 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 0.00 avg-zval = 0.00 2024/11/23-21:03:15.086438 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280570.sst: sumWaitingMem = 0.010 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0095 GB, waited 0.000 sec, Key+Value = 0.083 GB 2024/11/23-21:03:15.424683 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280570.sst): second pass time = 0.00's, 0.000'MB/sec, value only( 0.0% of KV) wait indexing time = 0.25's, remap KeyValue time = 0.34's, 55.877'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 4275.879'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 68.852'MB/sec write SST data time = 0.33's, 58.230'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = -nan'MB/sec zip my value time = 0.00's, unzip length = 0.000'GB zip my value throughput = 0.000'MB/sec zip pipeline throughput = 0.000'MB/sec entries = 3458056 avg-key = 24.00 avg-zkey = 1.68 avg-val = 0.00 avg-zval = 0.00 usrkeys = 3458056 avg-key = 16.00 avg-zkey = 1.68 avg-val = 8.43 avg-zval = 0.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 3458056 bytes = 13399984 } seq expand size = 25196504 multi value expand size = 3960448 cnt WriteAppend = 9 UnZipSize{ index = 0.0553 GB value = 0.0000 GB dict = 0.00 MB all = 0.0830 GB } __ZipSize{ index = 0.0058 GB value = 0.0000 GB dict = 0.00 MB all = 0.0192 GB } UnZip/Zip{ index = 9.5294 value = 0.0000 dict = 0.00 all = 4.3139 } Zip/UnZip{ index = 0.1049 value = inf dict = 0.00 all = 0.2318 } ---------------------------- total value len = 10.692167 GB avg = 0.182 KB (by entry num) total key len = 1.594204 GB avg = 0.027 KB total ukey len = 1.099836 GB avg = 0.015 KB total ukey num = 0.071978503 Billion total entry num = 0.058786232 Billion write speed all = 0.281260704 MB/sec (with seq num) write speed all = 0.270494786 MB/sec (without seq num) 2024/11/23-21:03:15.451792 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280570: 3458056 keys, 19238703 bytes, temperature: kWarm 2024/11/23-21:03:15.451861 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366995451826, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280570, "file_size": 19238703, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3671891195, "largest_seqno": 4259662020, "table_properties": {"data_size": 80, "index_size": 5806144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 82993344, "raw_average_key_size": 24, "raw_value_size": 0, "raw_average_value_size": 0, "num_data_blocks": 1, "num_entries": 3458056, "num_filter_entries": 0, "num_deletions": 990112, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 0, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZeroLengthBlobStore", "creation_time": 1732366990, "oldest_key_time": 0, "file_creation_time": 1732366990, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280570, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:03:15.487850 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280571.sst) /dev/shm/ToplingTemp/Topling-188284-TVyN2n 2024/11/23-21:03:18.330312 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280571.sst): first pass time = 2.84's, 94.439'MB/sec 2024/11/23-21:03:18.543516 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280571.sst): old prealloc_size = 295279001, real_size = 135547984 2024/11/23-21:03:18.545833 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280571.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 6036950 prefix = 4 raw-key = 0.0724 GB zip-key = 0.0101 GB avg-key = 12.00 avg-zkey = 1.68 raw-val = 0.1235 GB zip-val = 0.1239 GB avg-val = 20.47 avg-zval = 20.52 2024/11/23-21:03:18.555218 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280571.sst: sumWaitingMem = 0.017 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0174 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:03:19.692365 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280571.sst): second pass time = 0.00's,248588929.577'MB/sec, value only(46.0% of KV) wait indexing time = 0.21's, remap KeyValue time = 1.15's, 136.422'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 4411.824'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 2.894'MB/sec write SST data time = 0.62's, 250.771'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = -nan'MB/sec zip my value time = 0.00's, unzip length = 0.124'GB zip my value throughput = 248588929.577'MB/sec zip pipeline throughput = 0.000'MB/sec entries = 6036950 avg-key = 24.00 avg-zkey = 1.68 avg-val = 20.47 avg-zval = 20.52 usrkeys = 6036950 avg-key = 16.00 avg-zkey = 1.68 avg-val = 27.47 avg-zval = 20.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 6036950 bytes = 22638576 } seq expand size = 42258650 multi value expand size = 0 cnt WriteAppend = 15091 UnZipSize{ index = 0.0966 GB value = 0.1235 GB dict = 0.00 MB all = 0.2684 GB } __ZipSize{ index = 0.0101 GB value = 0.1239 GB dict = 0.00 MB all = 0.1567 GB } UnZip/Zip{ index = 9.5314 value = 0.9971 dict = 0.00 all = 1.7128 } Zip/UnZip{ index = 0.1049 value = 1.0029 dict = 0.00 all = 0.5839 } ---------------------------- total value len = 10.857974 GB avg = 0.168 KB (by entry num) total key len = 1.739091 GB avg = 0.027 KB total ukey len = 1.196427 GB avg = 0.015 KB total ukey num = 0.078015453 Billion total entry num = 0.064823182 Billion write speed all = 0.288344972 MB/sec (with seq num) write speed all = 0.276474627 MB/sec (without seq num) 2024/11/23-21:03:19.859729 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280571: 6036950 keys, 156726818 bytes, temperature: kWarm 2024/11/23-21:03:19.859800 139957483730496 EVENT_LOG_v1 {"time_micros": 1732366999859764, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280571, "file_size": 156726818, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3503605566, "largest_seqno": 4259662174, "table_properties": {"data_size": 123904576, "index_size": 10134016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 144886800, "raw_average_key_size": 24, "raw_value_size": 123548698, "raw_average_value_size": 20, "num_data_blocks": 1, "num_entries": 6036950, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732366995, "oldest_key_time": 0, "file_creation_time": 1732366995, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280571, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:03:19.922509 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280572.sst) /dev/shm/ToplingTemp/Topling-188284-nZoJfk 2024/11/23-21:03:25.066303 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280572.sst): first pass time = 5.14's, 52.187'MB/sec 2024/11/23-21:03:25.225838 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280572.sst): old prealloc_size = 295279001, real_size = 134832096 2024/11/23-21:03:25.228194 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280572.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 6070401 prefix = 4 raw-key = 0.0728 GB zip-key = 0.0102 GB avg-key = 12.00 avg-zkey = 1.68 raw-val = 0.1227 GB zip-val = 0.1231 GB avg-val = 20.22 avg-zval = 20.28 2024/11/23-21:03:25.235436 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280572.sst: sumWaitingMem = 0.017 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0175 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:03:26.420203 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280572.sst): second pass time = 0.00's,233801594.286'MB/sec, value only(45.7% of KV) wait indexing time = 0.16's, remap KeyValue time = 1.19's, 126.659'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 4368.752'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 2.739'MB/sec write SST data time = 0.64's, 237.117'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = -nan'MB/sec zip my value time = 0.00's, unzip length = 0.123'GB zip my value throughput = 233801594.286'MB/sec zip pipeline throughput = 0.000'MB/sec entries = 6070401 avg-key = 24.00 avg-zkey = 1.68 avg-val = 20.22 avg-zval = 20.28 usrkeys = 6070401 avg-key = 16.00 avg-zkey = 1.68 avg-val = 25.44 avg-zval = 20.28 TagRS{ kinds = RS_Key0_TagN bytes = 948600 } TagArray{ size = 4523457 bytes = 16962976 } seq expand size = 31664199 multi value expand size = 0 cnt WriteAppend = 14995 UnZipSize{ index = 0.0971 GB value = 0.1227 GB dict = 0.00 MB all = 0.2684 GB } __ZipSize{ index = 0.0102 GB value = 0.1231 GB dict = 0.00 MB all = 0.1513 GB } UnZip/Zip{ index = 9.5300 value = 0.9969 dict = 0.00 all = 1.7745 } Zip/UnZip{ index = 0.1049 value = 1.0031 dict = 0.00 all = 0.5635 } ---------------------------- total value len = 11.012384 GB avg = 0.155 KB (by entry num) total key len = 1.884780 GB avg = 0.027 KB total ukey len = 1.293554 GB avg = 0.015 KB total ukey num = 0.084085854 Billion total entry num = 0.070893583 Billion write speed all = 0.295168754 MB/sec (with seq num) write speed all = 0.282188803 MB/sec (without seq num) 2024/11/23-21:03:26.583610 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280572: 6070401 keys, 151275874 bytes, temperature: kWarm 2024/11/23-21:03:26.583688 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367006583645, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280572, "file_size": 151275874, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776303, "table_properties": {"data_size": 123122704, "index_size": 10191616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 145689624, "raw_average_key_size": 24, "raw_value_size": 122745837, "raw_average_value_size": 20, "num_data_blocks": 1, "num_entries": 6070401, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732366999, "oldest_key_time": 0, "file_creation_time": 1732366999, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280572, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:03:26.643907 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280573.sst) /dev/shm/ToplingTemp/Topling-188284-0eSQgl 2024/11/23-21:03:30.051934 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280573.sst): first pass time = 3.41's, 61.182'MB/sec 2024/11/23-21:03:30.202263 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280573.sst): old prealloc_size = 295279001, real_size = 105157920 2024/11/23-21:03:30.204053 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280573.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = ZipOffsetBlobStore usrkeys = 4695097 prefix = 4 raw-key = 0.0563 GB zip-key = 0.0079 GB avg-key = 12.00 avg-zkey = 1.68 raw-val = 0.0958 GB zip-val = 0.0961 GB avg-val = 20.41 avg-zval = 20.47 2024/11/23-21:03:30.210901 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280573.sst: sumWaitingMem = 0.013 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0135 GB, waited 0.000 sec, Key+Value = 0.209 GB 2024/11/23-21:03:31.095858 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280573.sst): second pass time = 0.00's,205191890.792'MB/sec, value only(46.0% of KV) wait indexing time = 0.15's, remap KeyValue time = 0.89's, 136.119'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 4451.851'MB/sec (index lex order gen) rebuild zvType time = 0.41's, 2.886'MB/sec write SST data time = 0.49's, 250.735'MB/sec dict compress time = 0.00's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = -nan'MB/sec zip my value time = 0.00's, unzip length = 0.096'GB zip my value throughput = 205191890.792'MB/sec zip pipeline throughput = 0.000'MB/sec entries = 4695097 avg-key = 24.00 avg-zkey = 1.68 avg-val = 20.41 avg-zval = 20.47 usrkeys = 4695097 avg-key = 16.00 avg-zkey = 1.68 avg-val = 27.41 avg-zval = 20.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 4695097 bytes = 17606624 } seq expand size = 32865679 multi value expand size = 0 cnt WriteAppend = 11707 UnZipSize{ index = 0.0751 GB value = 0.0958 GB dict = 0.00 MB all = 0.2085 GB } __ZipSize{ index = 0.0079 GB value = 0.0961 GB dict = 0.00 MB all = 0.1216 GB } UnZip/Zip{ index = 9.5309 value = 0.9971 dict = 0.00 all = 1.7142 } Zip/UnZip{ index = 0.1049 value = 1.0029 dict = 0.00 all = 0.5834 } ---------------------------- total value len = 11.141074 GB avg = 0.147 KB (by entry num) total key len = 1.997463 GB avg = 0.026 KB total ukey len = 1.368675 GB avg = 0.015 KB total ukey num = 0.088780951 Billion total entry num = 0.075588680 Billion write speed all = 0.300660714 MB/sec (with seq num) write speed all = 0.286822615 MB/sec (without seq num) 2024/11/23-21:03:31.230107 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280573: 4695097 keys, 121634146 bytes, temperature: kWarm 2024/11/23-21:03:31.230177 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367011230143, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280573, "file_size": 121634146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3236065447, "largest_seqno": 4259662017, "table_properties": {"data_size": 96102096, "index_size": 7881856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 112682328, "raw_average_key_size": 24, "raw_value_size": 95824613, "raw_average_value_size": 20, "num_data_blocks": 1, "num_entries": 4695097, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-ZipOffsetBlobStore", "creation_time": 1732367006, "oldest_key_time": 0, "file_creation_time": 1732367006, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280573, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:03:31.283342 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280574.sst) /dev/shm/ToplingTemp/Topling-188284-hTeOom 2024/11/23-21:03:33.312161 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280574.sst): first pass time = 2.03's, 132.314'MB/sec 2024/11/23-21:03:35.998424 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280574.sst): old prealloc_size = 295279001, real_size = 72587152 2024/11/23-21:03:35.999565 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280574.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1847856 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.2167 GB zip-val = 0.0686 GB avg-val = 117.27 avg-zval = 37.12 2024/11/23-21:03:36.000657 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280574.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:03:37.571381 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280574.sst): second pass time = 2.68's, 80.713'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.57's, 50.140'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3164.500'MB/sec (index lex order gen) rebuild zvType time = 1.31's, 0.353'MB/sec write SST data time = 0.26's, 300.415'MB/sec dict compress time = 0.00's, 2159556.000'MB/sec z-dict build time = 0.00's, sample length = 2.160'MB, throughput = 61284.863'MB/sec zip my value time = 2.68's, unzip length = 0.217'GB zip my value throughput = 80.713'MB/sec zip pipeline throughput = 80.713'MB/sec entries = 1847856 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.27 avg-zval = 37.12 usrkeys = 1847856 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.25 avg-zval = 37.12 TagRS{ kinds = RS_Key0_TagN bytes = 288840 } TagArray{ size = 1315616 bytes = 4275760 } seq expand size = 9209312 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0370 GB value = 0.2167 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0686 GB dict = 2.16 MB all = 0.0789 GB } UnZip/Zip{ index = 10.4502 value = 3.1594 dict = 1.00 all = 3.4036 } Zip/UnZip{ index = 0.0957 value = 0.3165 dict = 1.00 all = 0.2938 } ---------------------------- total value len = 11.366979 GB avg = 0.147 KB (by entry num) total key len = 2.049203 GB avg = 0.026 KB total ukey len = 1.405633 GB avg = 0.016 KB total ukey num = 0.090628807 Billion total entry num = 0.077436536 Billion write speed all = 0.306968819 MB/sec (with seq num) write speed all = 0.292794531 MB/sec (without seq num) 2024/11/23-21:03:37.662181 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280574: 1847856 keys, 78867222 bytes, temperature: kWarm 2024/11/23-21:03:37.662248 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367017662214, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280574, "file_size": 78867222, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700549, "table_properties": {"data_size": 68588480, "index_size": 3536512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51739968, "raw_average_key_size": 28, "raw_value_size": 216695612, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1847856, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367011, "oldest_key_time": 0, "file_creation_time": 1732367011, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280574, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:03:37.718768 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280575.sst) /dev/shm/ToplingTemp/Topling-188284-h9HqHl 2024/11/23-21:03:39.663878 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280575.sst): first pass time = 1.95's, 138.008'MB/sec 2024/11/23-21:03:42.860205 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280575.sst): old prealloc_size = 295279001, real_size = 72298352 2024/11/23-21:03:42.861335 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280575.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1853872 prefix = 4 raw-key = 0.0297 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.2165 GB zip-val = 0.0683 GB avg-val = 116.80 avg-zval = 36.83 2024/11/23-21:03:42.861379 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280575.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:03:44.273447 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280575.sst): second pass time = 3.19's, 67.775'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.41's, 52.371'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3260.262'MB/sec (index lex order gen) rebuild zvType time = 1.16's, 0.401'MB/sec write SST data time = 0.26's, 288.519'MB/sec dict compress time = 0.00's, 2159144.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 65087.390'MB/sec zip my value time = 3.19's, unzip length = 0.217'GB zip my value throughput = 67.775'MB/sec zip pipeline throughput = 67.775'MB/sec entries = 1853872 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.80 avg-zval = 36.83 usrkeys = 1853872 avg-key = 20.00 avg-zkey = 1.91 avg-val = 116.80 avg-zval = 36.83 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0371 GB value = 0.2165 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0683 GB dict = 2.16 MB all = 0.0740 GB } UnZip/Zip{ index = 10.4477 value = 3.1709 dict = 1.00 all = 3.6269 } Zip/UnZip{ index = 0.0957 value = 0.3154 dict = 1.00 all = 0.2757 } ---------------------------- total value len = 11.583506 GB avg = 0.146 KB (by entry num) total key len = 2.101111 GB avg = 0.026 KB total ukey len = 1.442710 GB avg = 0.016 KB total ukey num = 0.092482679 Billion total entry num = 0.079290408 Billion write speed all = 0.313062749 MB/sec (with seq num) write speed all = 0.298551346 MB/sec (without seq num) 2024/11/23-21:03:44.357948 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280575: 1853872 keys, 74011820 bytes, temperature: kWarm 2024/11/23-21:03:44.358035 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367024358000, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280575, "file_size": 74011820, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 68285856, "index_size": 3548864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51908416, "raw_average_key_size": 28, "raw_value_size": 216527124, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 1853872, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367017, "oldest_key_time": 0, "file_creation_time": 1732367017, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280575, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:03:44.413964 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280576.sst) /dev/shm/ToplingTemp/Topling-188284-RH86Km 2024/11/23-21:03:46.300593 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280576.sst): first pass time = 1.89's, 142.293'MB/sec 2024/11/23-21:03:49.736847 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280576.sst): old prealloc_size = 295279001, real_size = 72332112 2024/11/23-21:03:49.737973 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280576.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1853889 prefix = 4 raw-key = 0.0297 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2165 GB zip-val = 0.0683 GB avg-val = 116.80 avg-zval = 36.86 2024/11/23-21:03:49.738030 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280576.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:03:51.429129 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280576.sst): second pass time = 3.43's, 63.036'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.69's, 43.756'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3192.322'MB/sec (index lex order gen) rebuild zvType time = 1.29's, 0.360'MB/sec write SST data time = 0.41's, 182.599'MB/sec dict compress time = 0.00's, 2159540.000'MB/sec z-dict build time = 0.00's, sample length = 2.160'MB, throughput = 65731.418'MB/sec zip my value time = 3.43's, unzip length = 0.217'GB zip my value throughput = 63.036'MB/sec zip pipeline throughput = 63.036'MB/sec entries = 1853889 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 36.86 usrkeys = 1853889 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 36.86 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0371 GB value = 0.2165 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0683 GB dict = 2.16 MB all = 0.0740 GB } UnZip/Zip{ index = 10.5157 value = 3.1683 dict = 1.00 all = 3.6252 } Zip/UnZip{ index = 0.0951 value = 0.3156 dict = 1.00 all = 0.2758 } ---------------------------- total value len = 11.800033 GB avg = 0.145 KB (by entry num) total key len = 2.153020 GB avg = 0.027 KB total ukey len = 1.479788 GB avg = 0.016 KB total ukey num = 0.094336568 Billion total entry num = 0.081144297 Billion write speed all = 0.319151499 MB/sec (with seq num) write speed all = 0.304303237 MB/sec (without seq num) 2024/11/23-21:03:51.524196 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280576: 1853889 keys, 74046264 bytes, temperature: kWarm 2024/11/23-21:03:51.524265 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367031524231, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280576, "file_size": 74046264, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 68342528, "index_size": 3525952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51908892, "raw_average_key_size": 28, "raw_value_size": 216526690, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 1853889, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367024, "oldest_key_time": 0, "file_creation_time": 1732367024, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280576, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:03:51.588464 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280577.sst) /dev/shm/ToplingTemp/Topling-188284-6ssATk 2024/11/23-21:03:53.714407 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280577.sst): first pass time = 2.13's, 126.269'MB/sec 2024/11/23-21:03:56.074902 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280577.sst): old prealloc_size = 295279001, real_size = 55856928 2024/11/23-21:03:56.076030 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280577.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1838600 prefix = 4 raw-key = 0.0294 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.2170 GB zip-val = 0.0519 GB avg-val = 118.00 avg-zval = 28.20 2024/11/23-21:03:56.077010 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280577.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0048 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:03:57.723243 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280577.sst): second pass time = 2.36's, 91.964'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.65's, 37.353'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3196.950'MB/sec (index lex order gen) rebuild zvType time = 1.38's, 0.332'MB/sec write SST data time = 0.26's, 232.672'MB/sec dict compress time = 0.00's, 2162232.000'MB/sec z-dict build time = 0.00's, sample length = 2.162'MB, throughput = 64222.169'MB/sec zip my value time = 2.36's, unzip length = 0.217'GB zip my value throughput = 91.964'MB/sec zip pipeline throughput = 91.964'MB/sec entries = 1838600 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 28.20 usrkeys = 1838600 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.77 avg-zval = 28.20 TagRS{ kinds = RS_Key0_TagN bytes = 287400 } TagArray{ size = 990767 bytes = 3715392 } seq expand size = 6935369 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0368 GB value = 0.2170 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0519 GB dict = 2.16 MB all = 0.0616 GB } UnZip/Zip{ index = 10.3805 value = 4.1839 dict = 1.00 all = 4.3599 } Zip/UnZip{ index = 0.0963 value = 0.2390 dict = 1.00 all = 0.2294 } ---------------------------- total value len = 12.023923 GB avg = 0.145 KB (by entry num) total key len = 2.204501 GB avg = 0.027 KB total ukey len = 1.516560 GB avg = 0.016 KB total ukey num = 0.096175168 Billion total entry num = 0.082982897 Billion write speed all = 0.325403278 MB/sec (with seq num) write speed all = 0.310220763 MB/sec (without seq num) 2024/11/23-21:03:57.794112 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280577: 1838600 keys, 61569114 bytes, temperature: kWarm 2024/11/23-21:03:57.794186 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367037794152, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280577, "file_size": 61569114, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3919440447, "table_properties": {"data_size": 51854672, "index_size": 3542400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51480800, "raw_average_key_size": 28, "raw_value_size": 216954800, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 1838600, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367031, "oldest_key_time": 0, "file_creation_time": 1732367031, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280577, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:03:57.852093 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280578.sst) /dev/shm/ToplingTemp/Topling-188284-9rvpnk 2024/11/23-21:03:59.633185 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280578.sst): first pass time = 1.78's, 150.718'MB/sec 2024/11/23-21:04:02.169930 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280578.sst): old prealloc_size = 295279001, real_size = 60315328 2024/11/23-21:04:02.171025 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280578.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1850048 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2166 GB zip-val = 0.0563 GB avg-val = 117.10 avg-zval = 30.45 2024/11/23-21:04:02.173684 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280578.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:04:03.901402 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280578.sst): second pass time = 2.54's, 85.448'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.73's, 39.694'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3289.266'MB/sec (index lex order gen) rebuild zvType time = 1.44's, 0.322'MB/sec write SST data time = 0.29's, 235.198'MB/sec dict compress time = 0.00's, 2159198.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 72127.138'MB/sec zip my value time = 2.54's, unzip length = 0.217'GB zip my value throughput = 85.448'MB/sec zip pipeline throughput = 85.448'MB/sec entries = 1850048 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 30.45 usrkeys = 1850048 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 30.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1850048 bytes = 6706432 } seq expand size = 12950336 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0563 GB dict = 2.16 MB all = 0.0687 GB } UnZip/Zip{ index = 10.5174 value = 3.8455 dict = 1.00 all = 3.9057 } Zip/UnZip{ index = 0.0951 value = 0.2600 dict = 1.00 all = 0.2560 } ---------------------------- total value len = 12.253508 GB avg = 0.144 KB (by entry num) total key len = 2.256302 GB avg = 0.027 KB total ukey len = 1.553561 GB avg = 0.016 KB total ukey num = 0.098025216 Billion total entry num = 0.084832945 Billion write speed all = 0.331791676 MB/sec (with seq num) write speed all = 0.316272870 MB/sec (without seq num) 2024/11/23-21:04:03.981989 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280578: 1850048 keys, 68728944 bytes, temperature: kWarm 2024/11/23-21:04:03.982070 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367043982026, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280578, "file_size": 68728944, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3726348054, "largest_seqno": 4012776373, "table_properties": {"data_size": 56334576, "index_size": 3518080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51801344, "raw_average_key_size": 28, "raw_value_size": 216634144, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1850048, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367037, "oldest_key_time": 0, "file_creation_time": 1732367037, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280578, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:04:04.037237 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280579.sst) /dev/shm/ToplingTemp/Topling-188284-wqEUvn 2024/11/23-21:04:06.184282 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280579.sst): first pass time = 2.15's, 125.028'MB/sec 2024/11/23-21:04:08.685039 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280579.sst): old prealloc_size = 295279001, real_size = 60320048 2024/11/23-21:04:08.686140 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280579.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1850190 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.2166 GB zip-val = 0.0563 GB avg-val = 117.09 avg-zval = 30.44 2024/11/23-21:04:08.687712 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280579.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:04:10.369904 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280579.sst): second pass time = 2.50's, 86.667'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.68's, 40.933'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3273.943'MB/sec (index lex order gen) rebuild zvType time = 1.34's, 0.345'MB/sec write SST data time = 0.34's, 200.538'MB/sec dict compress time = 0.00's, 2159084.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 66865.407'MB/sec zip my value time = 2.50's, unzip length = 0.217'GB zip my value throughput = 86.667'MB/sec zip pipeline throughput = 86.667'MB/sec entries = 1850190 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.09 avg-zval = 30.44 usrkeys = 1850190 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.09 avg-zval = 30.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1850190 bytes = 6938224 } seq expand size = 12951330 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0563 GB dict = 2.16 MB all = 0.0690 GB } UnZip/Zip{ index = 10.4484 value = 3.8467 dict = 1.00 all = 3.8923 } Zip/UnZip{ index = 0.0957 value = 0.2600 dict = 1.00 all = 0.2569 } ---------------------------- total value len = 12.483089 GB avg = 0.144 KB (by entry num) total key len = 2.308107 GB avg = 0.027 KB total ukey len = 1.590565 GB avg = 0.016 KB total ukey num = 0.099875406 Billion total entry num = 0.086683135 Billion write speed all = 0.338176048 MB/sec (with seq num) write speed all = 0.322321125 MB/sec (without seq num) 2024/11/23-21:04:10.455279 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280579: 1850190 keys, 68965182 bytes, temperature: kWarm 2024/11/23-21:04:10.455368 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367050455320, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280579, "file_size": 68965182, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 4012776388, "table_properties": {"data_size": 56315744, "index_size": 3541568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51805320, "raw_average_key_size": 28, "raw_value_size": 216630172, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1850190, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367044, "oldest_key_time": 0, "file_creation_time": 1732367044, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280579, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:04:10.526373 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280580.sst) /dev/shm/ToplingTemp/Topling-188284-kDQWtl 2024/11/23-21:04:12.422174 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280580.sst): first pass time = 1.90's, 141.597'MB/sec 2024/11/23-21:04:14.993955 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280580.sst): old prealloc_size = 295279001, real_size = 60012288 2024/11/23-21:04:14.994989 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280580.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1850407 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2166 GB zip-val = 0.0560 GB avg-val = 117.07 avg-zval = 30.28 2024/11/23-21:04:14.996496 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280580.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:04:16.609374 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280580.sst): second pass time = 2.57's, 84.277'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.62's, 42.216'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3558.374'MB/sec (index lex order gen) rebuild zvType time = 1.35's, 0.342'MB/sec write SST data time = 0.26's, 261.713'MB/sec dict compress time = 0.00's, 2159292.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 69641.102'MB/sec zip my value time = 2.57's, unzip length = 0.217'GB zip my value throughput = 84.277'MB/sec zip pipeline throughput = 107.174'MB/sec entries = 1850407 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.07 avg-zval = 30.28 usrkeys = 1850407 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.07 avg-zval = 30.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1850407 bytes = 6476432 } seq expand size = 12952849 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0560 GB dict = 2.16 MB all = 0.0682 GB } UnZip/Zip{ index = 10.5164 value = 3.8662 dict = 1.00 all = 3.9362 } Zip/UnZip{ index = 0.0951 value = 0.2587 dict = 1.00 all = 0.2540 } ---------------------------- total value len = 12.712666 GB avg = 0.144 KB (by entry num) total key len = 2.359919 GB avg = 0.027 KB total ukey len = 1.627573 GB avg = 0.016 KB total ukey num = 0.101725813 Billion total entry num = 0.088533542 Billion write speed all = 0.344560371 MB/sec (with seq num) write speed all = 0.328369306 MB/sec (without seq num) 2024/11/23-21:04:16.688284 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280580: 1850407 keys, 68195854 bytes, temperature: kWarm 2024/11/23-21:04:16.688356 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367056688319, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280580, "file_size": 68195854, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3748831343, "largest_seqno": 4012776403, "table_properties": {"data_size": 56030384, "index_size": 3519104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51811396, "raw_average_key_size": 28, "raw_value_size": 216624142, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1850407, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367050, "oldest_key_time": 0, "file_creation_time": 1732367050, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280580, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:04:16.743347 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280581.sst) /dev/shm/ToplingTemp/Topling-188284-0wt2ck 2024/11/23-21:04:19.153225 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280581.sst): first pass time = 2.41's, 111.392'MB/sec 2024/11/23-21:04:21.488899 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280581.sst): old prealloc_size = 295279001, real_size = 59909168 2024/11/23-21:04:21.490037 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280581.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1850538 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2166 GB zip-val = 0.0559 GB avg-val = 117.06 avg-zval = 30.22 2024/11/23-21:04:21.492681 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280581.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:04:23.068784 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280581.sst): second pass time = 2.33's, 92.793'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.58's, 43.394'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3142.854'MB/sec (index lex order gen) rebuild zvType time = 1.31's, 0.352'MB/sec write SST data time = 0.26's, 259.549'MB/sec dict compress time = 0.00's, 2159174.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 59502.687'MB/sec zip my value time = 2.33's, unzip length = 0.217'GB zip my value throughput = 92.793'MB/sec zip pipeline throughput = 92.793'MB/sec entries = 1850538 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.06 avg-zval = 30.22 usrkeys = 1850538 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.06 avg-zval = 30.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1850538 bytes = 6939536 } seq expand size = 12953766 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0559 GB dict = 2.16 MB all = 0.0686 GB } UnZip/Zip{ index = 10.5171 value = 3.8733 dict = 1.00 all = 3.9156 } Zip/UnZip{ index = 0.0951 value = 0.2582 dict = 1.00 all = 0.2554 } ---------------------------- total value len = 12.942240 GB avg = 0.143 KB (by entry num) total key len = 2.411734 GB avg = 0.027 KB total ukey len = 1.664583 GB avg = 0.016 KB total ukey num = 0.103576351 Billion total entry num = 0.090384080 Billion write speed all = 0.350941128 MB/sec (with seq num) write speed all = 0.334414076 MB/sec (without seq num) 2024/11/23-21:04:23.147843 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280581: 1850538 keys, 68555736 bytes, temperature: kWarm 2024/11/23-21:04:23.147915 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367063147878, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280581, "file_size": 68555736, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 4012776423, "table_properties": {"data_size": 55927264, "index_size": 3519104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51815064, "raw_average_key_size": 28, "raw_value_size": 216620404, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1850538, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367056, "oldest_key_time": 0, "file_creation_time": 1732367056, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280581, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:04:23.205704 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280582.sst) /dev/shm/ToplingTemp/Topling-188284-lzFnDl 2024/11/23-21:04:24.817569 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280582.sst): first pass time = 1.61's, 166.541'MB/sec 2024/11/23-21:04:27.173204 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280582.sst): old prealloc_size = 295279001, real_size = 59645856 2024/11/23-21:04:27.174289 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280582.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1850255 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.2166 GB zip-val = 0.0556 GB avg-val = 117.08 avg-zval = 30.07 2024/11/23-21:04:27.175811 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280582.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:04:28.749248 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280582.sst): second pass time = 2.35's, 92.012'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.58's, 43.038'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3365.799'MB/sec (index lex order gen) rebuild zvType time = 1.31's, 0.352'MB/sec write SST data time = 0.26's, 260.596'MB/sec dict compress time = 0.00's, 2159302.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 63115.340'MB/sec zip my value time = 2.35's, unzip length = 0.217'GB zip my value throughput = 92.012'MB/sec zip pipeline throughput = 92.012'MB/sec entries = 1850255 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.08 avg-zval = 30.07 usrkeys = 1850255 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.08 avg-zval = 30.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1850255 bytes = 6475904 } seq expand size = 12951785 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0556 GB dict = 2.16 MB all = 0.0678 GB } UnZip/Zip{ index = 10.4507 value = 3.8932 dict = 1.00 all = 3.9575 } Zip/UnZip{ index = 0.0957 value = 0.2569 dict = 1.00 all = 0.2527 } ---------------------------- total value len = 13.171820 GB avg = 0.143 KB (by entry num) total key len = 2.463541 GB avg = 0.027 KB total ukey len = 1.701589 GB avg = 0.016 KB total ukey num = 0.105426606 Billion total entry num = 0.092234335 Billion write speed all = 0.357326319 MB/sec (with seq num) write speed all = 0.340463131 MB/sec (without seq num) 2024/11/23-21:04:28.827835 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280582: 1850255 keys, 67828952 bytes, temperature: kWarm 2024/11/23-21:04:28.827905 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367068827868, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280582, "file_size": 67828952, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759264366, "largest_seqno": 4012776433, "table_properties": {"data_size": 55642192, "index_size": 3540928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51807140, "raw_average_key_size": 28, "raw_value_size": 216628414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1850255, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367063, "oldest_key_time": 0, "file_creation_time": 1732367063, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280582, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:04:28.885231 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280583.sst) /dev/shm/ToplingTemp/Topling-188284-bkMoxm 2024/11/23-21:04:31.757373 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280583.sst): first pass time = 2.87's, 93.463'MB/sec 2024/11/23-21:04:34.085401 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280583.sst): old prealloc_size = 295279001, real_size = 58994048 2024/11/23-21:04:34.086538 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280583.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1849897 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.2166 GB zip-val = 0.0550 GB avg-val = 117.11 avg-zval = 29.73 2024/11/23-21:04:34.088141 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280583.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:04:35.625793 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280583.sst): second pass time = 2.33's, 93.107'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.54's, 43.911'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3226.810'MB/sec (index lex order gen) rebuild zvType time = 1.28's, 0.362'MB/sec write SST data time = 0.26's, 257.276'MB/sec dict compress time = 0.00's, 2158968.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 64518.065'MB/sec zip my value time = 2.33's, unzip length = 0.217'GB zip my value throughput = 93.107'MB/sec zip pipeline throughput = 93.107'MB/sec entries = 1849897 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.11 avg-zval = 29.73 usrkeys = 1849897 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.11 avg-zval = 29.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1849897 bytes = 6937136 } seq expand size = 12949279 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0550 GB dict = 2.16 MB all = 0.0676 GB } UnZip/Zip{ index = 10.4496 value = 3.9395 dict = 1.00 all = 3.9687 } Zip/UnZip{ index = 0.0957 value = 0.2538 dict = 1.00 all = 0.2520 } ---------------------------- total value len = 13.401408 GB avg = 0.142 KB (by entry num) total key len = 2.515338 GB avg = 0.027 KB total ukey len = 1.738586 GB avg = 0.016 KB total ukey num = 0.107276503 Billion total entry num = 0.094084232 Billion write speed all = 0.363699854 MB/sec (with seq num) write speed all = 0.346501153 MB/sec (without seq num) 2024/11/23-21:04:35.703193 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280583: 1849897 keys, 67638090 bytes, temperature: kWarm 2024/11/23-21:04:35.703264 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367075703228, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280583, "file_size": 67638090, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 4012776443, "table_properties": {"data_size": 54990768, "index_size": 3540608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51797116, "raw_average_key_size": 28, "raw_value_size": 216638434, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1849897, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367068, "oldest_key_time": 0, "file_creation_time": 1732367068, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280583, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:04:35.765713 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280584.sst) /dev/shm/ToplingTemp/Topling-188284-Rngtfk 2024/11/23-21:04:37.311810 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280584.sst): first pass time = 1.55's, 173.626'MB/sec 2024/11/23-21:04:39.643222 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280584.sst): old prealloc_size = 295279001, real_size = 60036464 2024/11/23-21:04:39.644300 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280584.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1850229 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2166 GB zip-val = 0.0561 GB avg-val = 117.08 avg-zval = 30.30 2024/11/23-21:04:39.645125 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280584.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:04:41.240260 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280584.sst): second pass time = 2.33's, 92.970'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.60's, 40.554'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3331.621'MB/sec (index lex order gen) rebuild zvType time = 1.33's, 0.347'MB/sec write SST data time = 0.26's, 246.594'MB/sec dict compress time = 0.00's, 2159244.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 63176.429'MB/sec zip my value time = 2.33's, unzip length = 0.217'GB zip my value throughput = 92.970'MB/sec zip pipeline throughput = 92.970'MB/sec entries = 1850229 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 30.30 usrkeys = 1850229 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.04 avg-zval = 30.30 TagRS{ kinds = RS_Key0_TagN bytes = 289200 } TagArray{ size = 780749 bytes = 2732640 } seq expand size = 5465243 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0561 GB dict = 2.16 MB all = 0.0648 GB } UnZip/Zip{ index = 10.5161 value = 3.8646 dict = 1.00 all = 4.1447 } Zip/UnZip{ index = 0.0951 value = 0.2588 dict = 1.00 all = 0.2413 } ---------------------------- total value len = 13.623502 GB avg = 0.142 KB (by entry num) total key len = 2.567145 GB avg = 0.027 KB total ukey len = 1.775591 GB avg = 0.016 KB total ukey num = 0.109126732 Billion total entry num = 0.095934461 Billion write speed all = 0.369911068 MB/sec (with seq num) write speed all = 0.352376392 MB/sec (without seq num) 2024/11/23-21:04:41.313558 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280584: 1850229 keys, 64765470 bytes, temperature: kWarm 2024/11/23-21:04:41.313628 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367081313593, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280584, "file_size": 64765470, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776458, "table_properties": {"data_size": 56054880, "index_size": 3518848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51806412, "raw_average_key_size": 28, "raw_value_size": 216629094, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1850229, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367075, "oldest_key_time": 0, "file_creation_time": 1732367075, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280584, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:04:41.370250 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280585.sst) /dev/shm/ToplingTemp/Topling-188284-4Q28Qm 2024/11/23-21:04:43.677864 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280585.sst): first pass time = 2.31's, 116.328'MB/sec 2024/11/23-21:04:45.997699 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280585.sst): old prealloc_size = 295279001, real_size = 57829872 2024/11/23-21:04:45.998866 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280585.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1850472 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2166 GB zip-val = 0.0538 GB avg-val = 117.06 avg-zval = 29.10 2024/11/23-21:04:46.000511 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280585.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:04:47.521234 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280585.sst): second pass time = 2.32's, 93.433'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.52's, 43.634'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3080.858'MB/sec (index lex order gen) rebuild zvType time = 1.26's, 0.367'MB/sec write SST data time = 0.26's, 255.173'MB/sec dict compress time = 0.00's, 2159098.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 61941.590'MB/sec zip my value time = 2.32's, unzip length = 0.217'GB zip my value throughput = 93.433'MB/sec zip pipeline throughput = 93.433'MB/sec entries = 1850472 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.06 avg-zval = 29.10 usrkeys = 1850472 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.06 avg-zval = 29.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1850472 bytes = 6939280 } seq expand size = 12953304 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0538 GB dict = 2.16 MB all = 0.0665 GB } UnZip/Zip{ index = 10.5177 value = 4.0228 dict = 1.00 all = 4.0381 } Zip/UnZip{ index = 0.0951 value = 0.2486 dict = 1.00 all = 0.2476 } ---------------------------- total value len = 13.853078 GB avg = 0.142 KB (by entry num) total key len = 2.618958 GB avg = 0.027 KB total ukey len = 1.812601 GB avg = 0.016 KB total ukey num = 0.110977204 Billion total entry num = 0.097784933 Billion write speed all = 0.376286019 MB/sec (with seq num) write speed all = 0.358415682 MB/sec (without seq num) 2024/11/23-21:04:47.597542 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280585: 1850472 keys, 66476108 bytes, temperature: kWarm 2024/11/23-21:04:47.597605 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367087597571, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280585, "file_size": 66476108, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 4012776470, "table_properties": {"data_size": 53848288, "index_size": 3518784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51813216, "raw_average_key_size": 28, "raw_value_size": 216622296, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1850472, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367081, "oldest_key_time": 0, "file_creation_time": 1732367081, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280585, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:04:47.655269 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280586.sst) /dev/shm/ToplingTemp/Topling-188284-SaZT3m 2024/11/23-21:04:49.320888 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280586.sst): first pass time = 1.67's, 161.165'MB/sec 2024/11/23-21:04:51.700728 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280586.sst): old prealloc_size = 295279001, real_size = 59182240 2024/11/23-21:04:51.701870 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280586.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1850199 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2166 GB zip-val = 0.0552 GB avg-val = 117.08 avg-zval = 29.83 2024/11/23-21:04:51.703443 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280586.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:04:53.255260 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280586.sst): second pass time = 2.38's, 91.090'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.55's, 43.484'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3145.608'MB/sec (index lex order gen) rebuild zvType time = 1.29's, 0.358'MB/sec write SST data time = 0.26's, 257.866'MB/sec dict compress time = 0.00's, 2159324.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 66279.628'MB/sec zip my value time = 2.38's, unzip length = 0.217'GB zip my value throughput = 91.090'MB/sec zip pipeline throughput = 91.090'MB/sec entries = 1850199 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 29.83 usrkeys = 1850199 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 29.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1850199 bytes = 6706992 } seq expand size = 12951393 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0552 GB dict = 2.16 MB all = 0.0676 GB } UnZip/Zip{ index = 10.5150 value = 3.9244 dict = 1.00 all = 3.9712 } Zip/UnZip{ index = 0.0951 value = 0.2548 dict = 1.00 all = 0.2518 } ---------------------------- total value len = 14.082659 GB avg = 0.141 KB (by entry num) total key len = 2.670763 GB avg = 0.027 KB total ukey len = 1.849604 GB avg = 0.016 KB total ukey num = 0.112827403 Billion total entry num = 0.099635132 Billion write speed all = 0.382663878 MB/sec (with seq num) write speed all = 0.364457800 MB/sec (without seq num) 2024/11/23-21:04:53.344069 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280586: 1850199 keys, 67596398 bytes, temperature: kWarm 2024/11/23-21:04:53.344153 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367093344115, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280586, "file_size": 67596398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3721406185, "largest_seqno": 4012776490, "table_properties": {"data_size": 55200336, "index_size": 3519168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51805572, "raw_average_key_size": 28, "raw_value_size": 216630014, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1850199, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367087, "oldest_key_time": 0, "file_creation_time": 1732367087, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280586, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:04:53.397660 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280587.sst) /dev/shm/ToplingTemp/Topling-188284-RG1cFl 2024/11/23-21:04:55.765351 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280587.sst): first pass time = 2.37's, 113.376'MB/sec 2024/11/23-21:04:58.297332 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280587.sst): old prealloc_size = 295279001, real_size = 61586848 2024/11/23-21:04:58.298462 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280587.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1850604 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0036 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.2166 GB zip-val = 0.0576 GB avg-val = 117.05 avg-zval = 31.10 2024/11/23-21:04:58.300092 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280587.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:04:59.923535 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280587.sst): second pass time = 2.53's, 85.598'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.63's, 43.350'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3219.848'MB/sec (index lex order gen) rebuild zvType time = 1.36's, 0.341'MB/sec write SST data time = 0.27's, 263.301'MB/sec dict compress time = 0.00's, 2159300.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 60474.430'MB/sec zip my value time = 2.53's, unzip length = 0.217'GB zip my value throughput = 85.598'MB/sec zip pipeline throughput = 85.598'MB/sec entries = 1850604 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.05 avg-zval = 31.10 usrkeys = 1850604 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.02 avg-zval = 31.10 TagRS{ kinds = RS_Key0_TagN bytes = 289240 } TagArray{ size = 1841160 bytes = 6904368 } seq expand size = 12888120 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0036 GB value = 0.0576 GB dict = 2.16 MB all = 0.0705 GB } UnZip/Zip{ index = 10.3843 value = 3.7634 dict = 1.00 all = 3.8079 } Zip/UnZip{ index = 0.0963 value = 0.2657 dict = 1.00 all = 0.2626 } ---------------------------- total value len = 14.312166 GB avg = 0.141 KB (by entry num) total key len = 2.722580 GB avg = 0.027 KB total ukey len = 1.886617 GB avg = 0.016 KB total ukey num = 0.114678007 Billion total entry num = 0.101485736 Billion write speed all = 0.389030322 MB/sec (with seq num) write speed all = 0.370488911 MB/sec (without seq num) 2024/11/23-21:05:00.010028 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280587: 1850604 keys, 70494742 bytes, temperature: kWarm 2024/11/23-21:05:00.010101 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367100010063, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280587, "file_size": 70494742, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4134951104, "table_properties": {"data_size": 57559760, "index_size": 3564224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51816912, "raw_average_key_size": 28, "raw_value_size": 216618648, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1850604, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367093, "oldest_key_time": 0, "file_creation_time": 1732367093, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280587, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:05:00.074550 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280588.sst) /dev/shm/ToplingTemp/Topling-188284-bxgu4m 2024/11/23-21:05:01.632372 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280588.sst): first pass time = 1.56's, 172.319'MB/sec 2024/11/23-21:05:03.933278 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280588.sst): old prealloc_size = 295279001, real_size = 60917072 2024/11/23-21:05:03.934353 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280588.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1843021 prefix = 4 raw-key = 0.0295 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2168 GB zip-val = 0.0570 GB avg-val = 117.65 avg-zval = 30.90 2024/11/23-21:05:03.936511 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280588.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0048 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:05:05.510679 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280588.sst): second pass time = 2.30's, 94.290'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.58's, 43.793'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3328.479'MB/sec (index lex order gen) rebuild zvType time = 1.31's, 0.351'MB/sec write SST data time = 0.26's, 263.256'MB/sec dict compress time = 0.00's, 2160312.000'MB/sec z-dict build time = 0.00's, sample length = 2.160'MB, throughput = 72654.604'MB/sec zip my value time = 2.30's, unzip length = 0.217'GB zip my value throughput = 94.290'MB/sec zip pipeline throughput = 94.290'MB/sec entries = 1843021 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.65 avg-zval = 30.90 usrkeys = 1843021 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.65 avg-zval = 30.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1843021 bytes = 6450592 } seq expand size = 12901147 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0369 GB value = 0.2168 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0570 GB dict = 2.16 MB all = 0.0691 GB } UnZip/Zip{ index = 10.5188 value = 3.8073 dict = 1.00 all = 3.8860 } Zip/UnZip{ index = 0.0951 value = 0.2627 dict = 1.00 all = 0.2573 } ---------------------------- total value len = 14.541898 GB avg = 0.141 KB (by entry num) total key len = 2.774185 GB avg = 0.027 KB total ukey len = 1.923477 GB avg = 0.017 KB total ukey num = 0.116521028 Billion total entry num = 0.103328757 Billion write speed all = 0.395404884 MB/sec (with seq num) write speed all = 0.376529163 MB/sec (without seq num) 2024/11/23-21:05:05.590925 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280588: 1843021 keys, 69077642 bytes, temperature: kWarm 2024/11/23-21:05:05.591025 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367105590959, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280588, "file_size": 69077642, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777810, "largest_seqno": 4259662056, "table_properties": {"data_size": 56951872, "index_size": 3504256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51604588, "raw_average_key_size": 28, "raw_value_size": 216830930, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1843021, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367100, "oldest_key_time": 0, "file_creation_time": 1732367100, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280588, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:05:05.648024 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280589.sst) /dev/shm/ToplingTemp/Topling-188284-ivZAio 2024/11/23-21:05:08.113882 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280589.sst): first pass time = 2.47's, 108.863'MB/sec 2024/11/23-21:05:10.415603 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280589.sst): old prealloc_size = 295279001, real_size = 60107808 2024/11/23-21:05:10.416749 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280589.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1843004 prefix = 4 raw-key = 0.0295 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2168 GB zip-val = 0.0561 GB avg-val = 117.65 avg-zval = 30.46 2024/11/23-21:05:10.418248 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280589.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0048 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:05:11.981800 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280589.sst): second pass time = 2.30's, 94.255'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.57's, 43.589'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3129.365'MB/sec (index lex order gen) rebuild zvType time = 1.31's, 0.353'MB/sec write SST data time = 0.26's, 264.320'MB/sec dict compress time = 0.00's, 2160436.000'MB/sec z-dict build time = 0.00's, sample length = 2.160'MB, throughput = 67657.397'MB/sec zip my value time = 2.30's, unzip length = 0.217'GB zip my value throughput = 94.255'MB/sec zip pipeline throughput = 94.255'MB/sec entries = 1843004 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.65 avg-zval = 30.46 usrkeys = 1843004 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.65 avg-zval = 30.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1843004 bytes = 6450528 } seq expand size = 12901028 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0369 GB value = 0.2168 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0561 GB dict = 2.16 MB all = 0.0683 GB } UnZip/Zip{ index = 10.5177 value = 3.8622 dict = 1.00 all = 3.9321 } Zip/UnZip{ index = 0.0951 value = 0.2589 dict = 1.00 all = 0.2543 } ---------------------------- total value len = 14.771631 GB avg = 0.140 KB (by entry num) total key len = 2.825789 GB avg = 0.027 KB total ukey len = 1.960337 GB avg = 0.017 KB total ukey num = 0.118364032 Billion total entry num = 0.105171761 Billion write speed all = 0.401769709 MB/sec (with seq num) write speed all = 0.382560153 MB/sec (without seq num) 2024/11/23-21:05:12.062137 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280589: 1843004 keys, 68268422 bytes, temperature: kWarm 2024/11/23-21:05:12.062211 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367112062173, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280589, "file_size": 68268422, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777827, "largest_seqno": 4259662073, "table_properties": {"data_size": 56142288, "index_size": 3504576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51604112, "raw_average_key_size": 28, "raw_value_size": 216831400, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1843004, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367105, "oldest_key_time": 0, "file_creation_time": 1732367105, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280589, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:05:12.118009 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280590.sst) /dev/shm/ToplingTemp/Topling-188284-NVQHzl 2024/11/23-21:05:14.348172 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280590.sst): first pass time = 2.23's, 120.368'MB/sec 2024/11/23-21:05:16.703895 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280590.sst): old prealloc_size = 295279001, real_size = 59087712 2024/11/23-21:05:16.705023 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280590.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1847429 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.2167 GB zip-val = 0.0551 GB avg-val = 117.30 avg-zval = 29.82 2024/11/23-21:05:16.706574 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280590.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0048 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:05:18.245940 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280590.sst): second pass time = 2.35's, 92.040'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.54's, 43.619'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3207.449'MB/sec (index lex order gen) rebuild zvType time = 1.28's, 0.360'MB/sec write SST data time = 0.26's, 262.254'MB/sec dict compress time = 0.00's, 2159102.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 59698.123'MB/sec zip my value time = 2.35's, unzip length = 0.217'GB zip my value throughput = 92.040'MB/sec zip pipeline throughput = 92.040'MB/sec entries = 1847429 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.30 avg-zval = 29.82 usrkeys = 1847429 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.30 avg-zval = 29.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1847429 bytes = 6466016 } seq expand size = 12932003 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0369 GB value = 0.2167 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0551 GB dict = 2.16 MB all = 0.0673 GB } UnZip/Zip{ index = 10.4500 value = 3.9337 dict = 1.00 all = 3.9909 } Zip/UnZip{ index = 0.0957 value = 0.2542 dict = 1.00 all = 0.2506 } ---------------------------- total value len = 15.001270 GB avg = 0.140 KB (by entry num) total key len = 2.877517 GB avg = 0.027 KB total ukey len = 1.997286 GB avg = 0.017 KB total ukey num = 0.120211461 Billion total entry num = 0.107019190 Billion write speed all = 0.408135287 MB/sec (with seq num) write speed all = 0.388591095 MB/sec (without seq num) 2024/11/23-21:05:18.324639 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280590: 1847429 keys, 67261456 bytes, temperature: kWarm 2024/11/23-21:05:18.324705 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367118324672, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280590, "file_size": 67261456, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777845, "largest_seqno": 4259662106, "table_properties": {"data_size": 55089936, "index_size": 3535744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51728012, "raw_average_key_size": 28, "raw_value_size": 216707462, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1847429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367112, "oldest_key_time": 0, "file_creation_time": 1732367112, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280590, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:05:18.379965 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280591.sst) /dev/shm/ToplingTemp/Topling-188284-jXsdFl 2024/11/23-21:05:19.749954 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280591.sst): first pass time = 1.37's, 195.946'MB/sec 2024/11/23-21:05:22.056872 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280591.sst): old prealloc_size = 295279001, real_size = 59126032 2024/11/23-21:05:22.057962 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280591.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1842840 prefix = 4 raw-key = 0.0295 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2168 GB zip-val = 0.0552 GB avg-val = 117.66 avg-zval = 29.93 2024/11/23-21:05:22.059450 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280591.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0048 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:05:23.611951 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280591.sst): second pass time = 2.31's, 94.043'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.56's, 43.269'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3256.723'MB/sec (index lex order gen) rebuild zvType time = 1.29's, 0.356'MB/sec write SST data time = 0.26's, 259.463'MB/sec dict compress time = 0.00's, 2160414.000'MB/sec z-dict build time = 0.00's, sample length = 2.160'MB, throughput = 66449.742'MB/sec zip my value time = 2.31's, unzip length = 0.217'GB zip my value throughput = 94.043'MB/sec zip pipeline throughput = 94.043'MB/sec entries = 1842840 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.66 avg-zval = 29.93 usrkeys = 1842840 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.66 avg-zval = 29.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1842840 bytes = 6449952 } seq expand size = 12899880 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0369 GB value = 0.2168 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0552 GB dict = 2.16 MB all = 0.0673 GB } UnZip/Zip{ index = 10.5179 value = 3.9310 dict = 1.00 all = 3.9895 } Zip/UnZip{ index = 0.0951 value = 0.2544 dict = 1.00 all = 0.2507 } ---------------------------- total value len = 15.231006 GB avg = 0.140 KB (by entry num) total key len = 2.929116 GB avg = 0.027 KB total ukey len = 2.034142 GB avg = 0.017 KB total ukey num = 0.122054301 Billion total entry num = 0.108862030 Billion write speed all = 0.414506812 MB/sec (with seq num) write speed all = 0.394628509 MB/sec (without seq num) 2024/11/23-21:05:23.689810 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280591: 1842840 keys, 67286128 bytes, temperature: kWarm 2024/11/23-21:05:23.689878 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367123689842, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280591, "file_size": 67286128, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777868, "largest_seqno": 4259662124, "table_properties": {"data_size": 55160960, "index_size": 3504192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51599520, "raw_average_key_size": 28, "raw_value_size": 216836048, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1842840, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367118, "oldest_key_time": 0, "file_creation_time": 1732367118, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280591, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:05:23.743413 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280592.sst) /dev/shm/ToplingTemp/Topling-188284-p7a4kn 2024/11/23-21:05:25.554157 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280592.sst): first pass time = 1.81's, 148.249'MB/sec 2024/11/23-21:05:27.906017 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280592.sst): old prealloc_size = 295279001, real_size = 60686848 2024/11/23-21:05:27.906997 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280592.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1842895 prefix = 4 raw-key = 0.0295 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2168 GB zip-val = 0.0567 GB avg-val = 117.66 avg-zval = 30.78 2024/11/23-21:05:27.908490 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280592.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0048 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:05:29.473230 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280592.sst): second pass time = 2.35's, 92.258'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.57's, 43.930'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3689.673'MB/sec (index lex order gen) rebuild zvType time = 1.31's, 0.353'MB/sec write SST data time = 0.26's, 263.928'MB/sec dict compress time = 0.00's, 2160358.000'MB/sec z-dict build time = 0.00's, sample length = 2.160'MB, throughput = 65232.140'MB/sec zip my value time = 2.35's, unzip length = 0.217'GB zip my value throughput = 92.258'MB/sec zip pipeline throughput = 92.258'MB/sec entries = 1842895 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.66 avg-zval = 30.78 usrkeys = 1842895 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.66 avg-zval = 30.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1842895 bytes = 6450144 } seq expand size = 12900265 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0369 GB value = 0.2168 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0567 GB dict = 2.16 MB all = 0.0688 GB } UnZip/Zip{ index = 10.5180 value = 3.8228 dict = 1.00 all = 3.8990 } Zip/UnZip{ index = 0.0951 value = 0.2616 dict = 1.00 all = 0.2565 } ---------------------------- total value len = 15.460741 GB avg = 0.140 KB (by entry num) total key len = 2.980718 GB avg = 0.027 KB total ukey len = 2.071000 GB avg = 0.017 KB total ukey num = 0.123897196 Billion total entry num = 0.110704925 Billion write speed all = 0.420872026 MB/sec (with seq num) write speed all = 0.400659913 MB/sec (without seq num) 2024/11/23-21:05:29.551060 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280592: 1842895 keys, 68847096 bytes, temperature: kWarm 2024/11/23-21:05:29.551149 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367129551095, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280592, "file_size": 68847096, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777888, "largest_seqno": 4259662135, "table_properties": {"data_size": 56721712, "index_size": 3504256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51601060, "raw_average_key_size": 28, "raw_value_size": 216834462, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1842895, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367123, "oldest_key_time": 0, "file_creation_time": 1732367123, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280592, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:05:29.606817 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280593.sst) /dev/shm/ToplingTemp/Topling-188284-4WcaFn 2024/11/23-21:05:31.973094 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280593.sst): first pass time = 2.37's, 113.444'MB/sec 2024/11/23-21:05:34.294418 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280593.sst): old prealloc_size = 295279001, real_size = 60739408 2024/11/23-21:05:34.295548 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280593.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1843106 prefix = 4 raw-key = 0.0295 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2168 GB zip-val = 0.0568 GB avg-val = 117.64 avg-zval = 30.80 2024/11/23-21:05:34.297038 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280593.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0048 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:05:35.861951 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280593.sst): second pass time = 2.32's, 93.463'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.57's, 43.956'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3163.578'MB/sec (index lex order gen) rebuild zvType time = 1.31's, 0.352'MB/sec write SST data time = 0.26's, 267.372'MB/sec dict compress time = 0.00's, 2160628.000'MB/sec z-dict build time = 0.00's, sample length = 2.161'MB, throughput = 61204.124'MB/sec zip my value time = 2.32's, unzip length = 0.217'GB zip my value throughput = 93.463'MB/sec zip pipeline throughput = 93.463'MB/sec entries = 1843106 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.64 avg-zval = 30.80 usrkeys = 1843106 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.64 avg-zval = 30.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1843106 bytes = 6450880 } seq expand size = 12901742 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0369 GB value = 0.2168 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0568 GB dict = 2.16 MB all = 0.0689 GB } UnZip/Zip{ index = 10.5198 value = 3.8191 dict = 1.00 all = 3.8960 } Zip/UnZip{ index = 0.0951 value = 0.2618 dict = 1.00 all = 0.2567 } ---------------------------- total value len = 15.690471 GB avg = 0.139 KB (by entry num) total key len = 3.032324 GB avg = 0.027 KB total ukey len = 2.107862 GB avg = 0.017 KB total ukey num = 0.125740302 Billion total entry num = 0.112548031 Billion write speed all = 0.427230428 MB/sec (with seq num) write speed all = 0.406684803 MB/sec (without seq num) 2024/11/23-21:05:35.941795 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280593: 1843106 keys, 68900614 bytes, temperature: kWarm 2024/11/23-21:05:35.941865 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367135941829, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280593, "file_size": 68900614, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777899, "largest_seqno": 4259662145, "table_properties": {"data_size": 56774400, "index_size": 3504064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51606968, "raw_average_key_size": 28, "raw_value_size": 216828492, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1843106, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367129, "oldest_key_time": 0, "file_creation_time": 1732367129, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280593, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:05:35.998815 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280594.sst) /dev/shm/ToplingTemp/Topling-188284-hdkM4k 2024/11/23-21:05:37.633444 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280594.sst): first pass time = 1.63's, 164.221'MB/sec 2024/11/23-21:05:39.964819 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280594.sst): old prealloc_size = 295279001, real_size = 59794560 2024/11/23-21:05:39.965889 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280594.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1842939 prefix = 4 raw-key = 0.0295 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2168 GB zip-val = 0.0558 GB avg-val = 117.66 avg-zval = 30.29 2024/11/23-21:05:39.967359 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280594.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0048 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:05:41.524265 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280594.sst): second pass time = 2.33's, 93.059'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.56's, 43.577'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3351.464'MB/sec (index lex order gen) rebuild zvType time = 1.30's, 0.355'MB/sec write SST data time = 0.26's, 261.936'MB/sec dict compress time = 0.00's, 2160518.000'MB/sec z-dict build time = 0.00's, sample length = 2.161'MB, throughput = 59549.571'MB/sec zip my value time = 2.33's, unzip length = 0.217'GB zip my value throughput = 93.059'MB/sec zip pipeline throughput = 93.059'MB/sec entries = 1842939 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.66 avg-zval = 30.29 usrkeys = 1842939 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.66 avg-zval = 30.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1842939 bytes = 6450304 } seq expand size = 12900573 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0369 GB value = 0.2168 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0558 GB dict = 2.16 MB all = 0.0680 GB } UnZip/Zip{ index = 10.5177 value = 3.8839 dict = 1.00 all = 3.9502 } Zip/UnZip{ index = 0.0951 value = 0.2575 dict = 1.00 all = 0.2532 } ---------------------------- total value len = 15.920205 GB avg = 0.139 KB (by entry num) total key len = 3.083927 GB avg = 0.027 KB total ukey len = 2.144721 GB avg = 0.017 KB total ukey num = 0.127583241 Billion total entry num = 0.114390970 Billion write speed all = 0.433594138 MB/sec (with seq num) write speed all = 0.412714782 MB/sec (without seq num) 2024/11/23-21:05:41.603599 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280594: 1842939 keys, 67955096 bytes, temperature: kWarm 2024/11/23-21:05:41.603669 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367141603633, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280594, "file_size": 67955096, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777911, "largest_seqno": 4259662156, "table_properties": {"data_size": 55829168, "index_size": 3504448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51602292, "raw_average_key_size": 28, "raw_value_size": 216833242, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1842939, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367135, "oldest_key_time": 0, "file_creation_time": 1732367135, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280594, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:05:41.668509 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280595.sst) /dev/shm/ToplingTemp/Topling-188284-x3WR1j 2024/11/23-21:05:43.559075 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280595.sst): first pass time = 1.89's, 141.990'MB/sec 2024/11/23-21:05:45.934193 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280595.sst): old prealloc_size = 295279001, real_size = 59173296 2024/11/23-21:05:45.935286 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280595.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1848505 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.2167 GB zip-val = 0.0552 GB avg-val = 117.22 avg-zval = 29.87 2024/11/23-21:05:45.937435 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280595.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:05:47.479765 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280595.sst): second pass time = 2.37's, 91.276'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.55's, 43.454'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3263.263'MB/sec (index lex order gen) rebuild zvType time = 1.28's, 0.362'MB/sec write SST data time = 0.27's, 250.898'MB/sec dict compress time = 0.00's, 2159558.000'MB/sec z-dict build time = 0.00's, sample length = 2.160'MB, throughput = 67046.197'MB/sec zip my value time = 2.37's, unzip length = 0.217'GB zip my value throughput = 91.276'MB/sec zip pipeline throughput = 91.276'MB/sec entries = 1848505 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.22 avg-zval = 29.87 usrkeys = 1848505 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.69 avg-zval = 29.87 TagRS{ kinds = RS_Key0_TagN bytes = 288920 } TagArray{ size = 1709368 bytes = 5982800 } seq expand size = 11965576 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0370 GB value = 0.2167 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0552 GB dict = 2.16 MB all = 0.0672 GB } UnZip/Zip{ index = 10.5874 value = 3.9240 dict = 1.00 all = 3.9969 } Zip/UnZip{ index = 0.0945 value = 0.2548 dict = 1.00 all = 0.2502 } ---------------------------- total value len = 16.148848 GB avg = 0.139 KB (by entry num) total key len = 3.135685 GB avg = 0.027 KB total ukey len = 2.181691 GB avg = 0.017 KB total ukey num = 0.129431746 Billion total entry num = 0.116239475 Billion write speed all = 0.439931931 MB/sec (with seq num) write speed all = 0.418718057 MB/sec (without seq num) 2024/11/23-21:05:47.557939 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280595: 1848505 keys, 67160344 bytes, temperature: kWarm 2024/11/23-21:05:47.558025 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367147557991, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280595, "file_size": 67160344, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4259662188, "table_properties": {"data_size": 55219104, "index_size": 3491904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51758140, "raw_average_key_size": 28, "raw_value_size": 216677398, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1848505, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367141, "oldest_key_time": 0, "file_creation_time": 1732367141, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280595, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:05:47.613083 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280596.sst) /dev/shm/ToplingTemp/Topling-188284-s89Lon 2024/11/23-21:05:49.414745 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280596.sst): first pass time = 1.80's, 148.996'MB/sec 2024/11/23-21:05:52.540710 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280596.sst): old prealloc_size = 295279001, real_size = 72233936 2024/11/23-21:05:52.541837 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280596.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1850858 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2166 GB zip-val = 0.0683 GB avg-val = 117.03 avg-zval = 36.88 2024/11/23-21:05:52.542386 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280596.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:05:53.993487 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280596.sst): second pass time = 3.12's, 69.324'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.45's, 52.057'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3188.315'MB/sec (index lex order gen) rebuild zvType time = 1.19's, 0.388'MB/sec write SST data time = 0.26's, 292.223'MB/sec dict compress time = 0.00's, 2159014.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 63007.471'MB/sec zip my value time = 3.12's, unzip length = 0.217'GB zip my value throughput = 69.324'MB/sec zip pipeline throughput = 69.324'MB/sec entries = 1850858 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.03 avg-zval = 36.88 usrkeys = 1850858 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.43 avg-zval = 36.88 TagRS{ kinds = RS_Key0_TagN bytes = 289280 } TagArray{ size = 370065 bytes = 1387760 } seq expand size = 2590455 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0683 GB dict = 2.16 MB all = 0.0756 GB } UnZip/Zip{ index = 10.5172 value = 3.1737 dict = 1.00 all = 3.5496 } Zip/UnZip{ index = 0.0951 value = 0.3151 dict = 1.00 all = 0.2817 } ---------------------------- total value len = 16.368050 GB avg = 0.139 KB (by entry num) total key len = 3.187509 GB avg = 0.027 KB total ukey len = 2.218709 GB avg = 0.017 KB total ukey num = 0.131282604 Billion total entry num = 0.118090333 Billion write speed all = 0.446048481 MB/sec (with seq num) write speed all = 0.424500025 MB/sec (without seq num) 2024/11/23-21:05:54.080573 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280596: 1850858 keys, 75625143 bytes, temperature: kWarm 2024/11/23-21:05:54.080645 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367154080609, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280596, "file_size": 75625143, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3973738401, "table_properties": {"data_size": 68251328, "index_size": 3519680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51824024, "raw_average_key_size": 28, "raw_value_size": 216611564, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 1850858, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367147, "oldest_key_time": 0, "file_creation_time": 1732367147, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280596, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:05:54.141410 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280597.sst) /dev/shm/ToplingTemp/Topling-188284-627Tol 2024/11/23-21:05:56.204267 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280597.sst): first pass time = 2.06's, 130.130'MB/sec 2024/11/23-21:05:58.576851 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280597.sst): old prealloc_size = 295279001, real_size = 60841456 2024/11/23-21:05:58.577965 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280597.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1862415 prefix = 4 raw-key = 0.0298 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2163 GB zip-val = 0.0568 GB avg-val = 116.13 avg-zval = 30.52 2024/11/23-21:05:58.580565 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280597.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:06:00.239211 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280597.sst): second pass time = 2.37's, 91.215'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.66's, 41.684'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3224.619'MB/sec (index lex order gen) rebuild zvType time = 1.40's, 0.333'MB/sec write SST data time = 0.26's, 261.954'MB/sec dict compress time = 0.00's, 2155758.000'MB/sec z-dict build time = 0.00's, sample length = 2.156'MB, throughput = 67693.211'MB/sec zip my value time = 2.37's, unzip length = 0.216'GB zip my value throughput = 91.215'MB/sec zip pipeline throughput = 91.215'MB/sec entries = 1862415 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 30.52 usrkeys = 1862415 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 30.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1862415 bytes = 6751264 } seq expand size = 13036905 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0372 GB value = 0.2163 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0568 GB dict = 2.16 MB all = 0.0693 GB } UnZip/Zip{ index = 10.5182 value = 3.8056 dict = 1.00 all = 3.8739 } Zip/UnZip{ index = 0.0951 value = 0.2628 dict = 1.00 all = 0.2581 } ---------------------------- total value len = 16.597375 GB avg = 0.138 KB (by entry num) total key len = 3.239657 GB avg = 0.027 KB total ukey len = 2.255957 GB avg = 0.017 KB total ukey num = 0.133145019 Billion total entry num = 0.119952748 Billion write speed all = 0.452404217 MB/sec (with seq num) write speed all = 0.430519036 MB/sec (without seq num) 2024/11/23-21:06:00.322183 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280597: 1862415 keys, 69293376 bytes, temperature: kWarm 2024/11/23-21:06:00.322259 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367160322221, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280597, "file_size": 69293376, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3509327862, "largest_seqno": 4012776072, "table_properties": {"data_size": 56834336, "index_size": 3541312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 52147620, "raw_average_key_size": 28, "raw_value_size": 216287842, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 1862415, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367154, "oldest_key_time": 0, "file_creation_time": 1732367154, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280597, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:00.386376 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280598.sst) /dev/shm/ToplingTemp/Topling-188284-22jcho 2024/11/23-21:06:02.559323 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280598.sst): first pass time = 2.17's, 123.538'MB/sec 2024/11/23-21:06:04.928428 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280598.sst): old prealloc_size = 295279001, real_size = 62270912 2024/11/23-21:06:04.929535 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280598.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1852312 prefix = 4 raw-key = 0.0296 GB zip-key = 0.0036 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.2166 GB zip-val = 0.0582 GB avg-val = 116.92 avg-zval = 31.44 2024/11/23-21:06:04.930832 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280598.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:06:06.551431 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280598.sst): second pass time = 2.37's, 91.465'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.62's, 42.936'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3286.692'MB/sec (index lex order gen) rebuild zvType time = 1.36's, 0.341'MB/sec write SST data time = 0.27's, 262.863'MB/sec dict compress time = 0.00's, 2158618.000'MB/sec z-dict build time = 0.00's, sample length = 2.159'MB, throughput = 53745.095'MB/sec zip my value time = 2.37's, unzip length = 0.217'GB zip my value throughput = 91.465'MB/sec zip pipeline throughput = 91.465'MB/sec entries = 1852312 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.92 avg-zval = 31.44 usrkeys = 1852312 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.57 avg-zval = 31.44 TagRS{ kinds = RS_Key0_TagN bytes = 289520 } TagArray{ size = 1494463 bytes = 5417440 } seq expand size = 10461241 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0370 GB value = 0.2166 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0036 GB value = 0.0582 GB dict = 2.16 MB all = 0.0697 GB } UnZip/Zip{ index = 10.3818 value = 3.7186 dict = 1.00 all = 3.8522 } Zip/UnZip{ index = 0.0963 value = 0.2689 dict = 1.00 all = 0.2596 } ---------------------------- total value len = 16.824407 GB avg = 0.138 KB (by entry num) total key len = 3.291521 GB avg = 0.027 KB total ukey len = 2.293003 GB avg = 0.017 KB total ukey num = 0.134997331 Billion total entry num = 0.121805060 Billion write speed all = 0.458698716 MB/sec (with seq num) write speed all = 0.436478782 MB/sec (without seq num) 2024/11/23-21:06:06.631844 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280598: 1852312 keys, 69684012 bytes, temperature: kWarm 2024/11/23-21:06:06.631915 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367166631879, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280598, "file_size": 69684012, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776108, "table_properties": {"data_size": 58239280, "index_size": 3568384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 51864736, "raw_average_key_size": 28, "raw_value_size": 216570780, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 1852312, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367160, "oldest_key_time": 0, "file_creation_time": 1732367160, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280598, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:06.688347 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280599.sst) /dev/shm/ToplingTemp/Topling-188284-WBCYkn 2024/11/23-21:06:08.387971 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280599.sst): first pass time = 1.70's, 157.942'MB/sec 2024/11/23-21:06:10.772438 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280599.sst): old prealloc_size = 295279001, real_size = 60382656 2024/11/23-21:06:10.773465 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280599.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1862993 prefix = 4 raw-key = 0.0298 GB zip-key = 0.0036 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.2163 GB zip-val = 0.0564 GB avg-val = 116.09 avg-zval = 30.25 2024/11/23-21:06:10.774902 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280599.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:06:12.377317 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280599.sst): second pass time = 2.38's, 90.750'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.60's, 42.643'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3557.624'MB/sec (index lex order gen) rebuild zvType time = 1.34's, 0.348'MB/sec write SST data time = 0.26's, 258.587'MB/sec dict compress time = 0.00's, 2155256.000'MB/sec z-dict build time = 0.00's, sample length = 2.155'MB, throughput = 68273.441'MB/sec zip my value time = 2.38's, unzip length = 0.216'GB zip my value throughput = 90.750'MB/sec zip pipeline throughput = 90.750'MB/sec entries = 1862993 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.09 avg-zval = 30.25 usrkeys = 1862993 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.16 avg-zval = 30.25 TagRS{ kinds = RS_Key0_TagN bytes = 291200 } TagArray{ size = 1616412 bytes = 6061552 } seq expand size = 11314884 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0373 GB value = 0.2163 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0036 GB value = 0.0564 GB dict = 2.16 MB all = 0.0684 GB } UnZip/Zip{ index = 10.4507 value = 3.8379 dict = 1.00 all = 3.9225 } Zip/UnZip{ index = 0.0957 value = 0.2606 dict = 1.00 all = 0.2549 } ---------------------------- total value len = 17.051993 GB avg = 0.138 KB (by entry num) total key len = 3.343685 GB avg = 0.027 KB total ukey len = 2.330263 GB avg = 0.017 KB total ukey num = 0.136860324 Billion total entry num = 0.123668053 Billion write speed all = 0.465016021 MB/sec (with seq num) write speed all = 0.442459233 MB/sec (without seq num) 2024/11/23-21:06:12.429431 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280599: 1862993 keys, 68435338 bytes, temperature: kWarm 2024/11/23-21:06:12.429498 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367172429463, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280599, "file_size": 68435338, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776133, "table_properties": {"data_size": 56351408, "index_size": 3565312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 52163804, "raw_average_key_size": 28, "raw_value_size": 216271662, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 1862993, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367166, "oldest_key_time": 0, "file_creation_time": 1732367166, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280599, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:12.485624 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280600.sst) /dev/shm/ToplingTemp/Topling-188284-KgXTzm 2024/11/23-21:06:14.379065 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280600.sst): first pass time = 1.89's, 141.774'MB/sec 2024/11/23-21:06:16.759912 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280600.sst): old prealloc_size = 295279001, real_size = 61973056 2024/11/23-21:06:16.760930 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280600.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1859694 prefix = 4 raw-key = 0.0298 GB zip-key = 0.0035 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.2164 GB zip-val = 0.0580 GB avg-val = 116.34 avg-zval = 31.17 2024/11/23-21:06:16.762233 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280600.sst: sumWaitingMem = 0.005 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0049 GB, waited 0.000 sec, Key+Value = 0.268 GB 2024/11/23-21:06:18.444146 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280600.sst): second pass time = 2.38's, 90.927'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.68's, 41.141'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3552.202'MB/sec (index lex order gen) rebuild zvType time = 1.42's, 0.328'MB/sec write SST data time = 0.27's, 259.800'MB/sec dict compress time = 0.00's, 2157480.000'MB/sec z-dict build time = 0.00's, sample length = 2.157'MB, throughput = 60048.429'MB/sec zip my value time = 2.38's, unzip length = 0.216'GB zip my value throughput = 90.927'MB/sec zip pipeline throughput = 90.927'MB/sec entries = 1859694 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.34 avg-zval = 31.17 usrkeys = 1859694 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.87 avg-zval = 31.17 TagRS{ kinds = RS_Key0_TagN bytes = 290680 } TagArray{ size = 1468552 bytes = 5323520 } seq expand size = 10279864 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0372 GB value = 0.2164 GB dict = 2.16 MB all = 0.2684 GB } __ZipSize{ index = 0.0035 GB value = 0.0580 GB dict = 2.16 MB all = 0.0693 GB } UnZip/Zip{ index = 10.5181 value = 3.7322 dict = 1.00 all = 3.8741 } Zip/UnZip{ index = 0.0951 value = 0.2679 dict = 1.00 all = 0.2581 } ---------------------------- total value len = 17.278637 GB avg = 0.138 KB (by entry num) total key len = 3.395757 GB avg = 0.027 KB total ukey len = 2.367457 GB avg = 0.017 KB total ukey num = 0.138720018 Billion total entry num = 0.125527747 Billion write speed all = 0.471305465 MB/sec (with seq num) write speed all = 0.448412639 MB/sec (without seq num) 2024/11/23-21:06:18.534969 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280600: 1859694 keys, 69290362 bytes, temperature: kWarm 2024/11/23-21:06:18.535050 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367178535014, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280600, "file_size": 69290362, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 57971760, "index_size": 3536192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 52071432, "raw_average_key_size": 28, "raw_value_size": 216364052, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 1859694, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367172, "oldest_key_time": 0, "file_creation_time": 1732367172, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280600, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:18.590512 139957483730496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280601.sst) /dev/shm/ToplingTemp/Topling-188284-w6fcfn 2024/11/23-21:06:19.113740 139957483730496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280601.sst): first pass time = 0.52's, 255.298'MB/sec 2024/11/23-21:06:20.407713 139957483730496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280601.sst): old prealloc_size = 295279001, real_size = 30735680 2024/11/23-21:06:20.408316 139957483730496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280601.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921100 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0287 GB avg-val = 117.01 avg-zval = 31.20 2024/11/23-21:06:20.408725 139957483730496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280601.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:06:21.221005 139957483730496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00/280601.sst): second pass time = 1.29's, 83.384'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 40.659'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3041.018'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 256.968'MB/sec dict compress time = 0.00's, 1077320.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 27930.105'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.384'MB/sec zip pipeline throughput = 83.384'MB/sec entries = 921100 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.01 avg-zval = 31.20 usrkeys = 921100 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.02 avg-zval = 31.20 TagRS{ kinds = RS_Key0_TagN bytes = 144040 } TagArray{ size = 395769 bytes = 1335728 } seq expand size = 2770383 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1078 GB dict = 1.08 MB all = 0.1336 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0331 GB } UnZip/Zip{ index = 10.4473 value = 3.7499 dict = 1.00 all = 4.0395 } Zip/UnZip{ index = 0.0957 value = 0.2667 dict = 1.00 all = 0.2476 } ---------------------------- total value len = 17.389188 GB avg = 0.138 KB (by entry num) total key len = 3.421547 GB avg = 0.027 KB total ukey len = 2.385879 GB avg = 0.017 KB total ukey num = 0.139641118 Billion total entry num = 0.126448847 Billion write speed all = 0.474383556 MB/sec (with seq num) write speed all = 0.451324206 MB/sec (without seq num) 2024/11/23-21:06:21.267687 139957483730496 [compaction_job.cc:1948] [default] [JOB 35741] Generated table #280601: 921100 keys, 33066455 bytes, temperature: kWarm 2024/11/23-21:06:21.267753 139957483730496 EVENT_LOG_v1 {"time_micros": 1732367181267720, "cf_name": "default", "job": 35741, "event": "table_file_creation", "file_number": 280601, "file_size": 33066455, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776176, "table_properties": {"data_size": 28741872, "index_size": 1763328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25790800, "raw_average_key_size": 28, "raw_value_size": 107780352, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921100, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367178, "oldest_key_time": 0, "file_creation_time": 1732367178, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280601, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:21.295039 139957483730496 [compaction_job.cc:703] job-35741: subcompact[0], size: 3.955808 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35741/att-00 [280560,280561,280562,280563,280564,280565,280566,280567,280568,280569,280570,280571,280572,280573,280574,280575,280576,280577,280578,280579,280580,280581,280582,280583,280584,280585,280586,280587,280588,280589,280590,280591,280592,280593,280594,280595,280596,280597,280598,280599,280600,280601] 2024/11/23-21:06:21.820879 139957483730496 (Original Log Time 2024/11/23-21:02:12.257262) EVENT_LOG_v1 {"time_micros": 1732366932257206, "job": 35741, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L3": [280463, 280457, 280506, 280507, 280508, 280509, 280467, 280459, 280461, 280462, 262667, 262668, 262669, 262670, 262671, 264939, 280500, 280501, 280502, 280503, 280537], "files_L4": [280425, 280426, 267485, 262338, 274572, 274573, 271322, 271323, 234406, 270325, 213920, 227563, 237538, 271321, 280231, 280427, 275661, 279539, 279540, 277299, 277300, 206487, 244184, 245367, 247941, 255717, 262472, 262473, 258439, 259954, 259955, 260685, 260686, 259249, 259250, 276084, 276085, 275122, 280015, 280016, 279794, 279795, 276655, 276656, 264775, 261882, 262673, 263783, 271154, 272081, 266193, 266194, 267895, 269339], "score": -1, "input_data_size": 4463661836, "oldest_snapshot_seqno": 4260463563} 2024/11/23-21:06:21.820883 139957483730496 (Original Log Time 2024/11/23-21:06:21.819275) [compaction_job.cc:2022] [default] [JOB 35741] Compacted 21@3 + 54@4 files to L4 => 3955807534 bytes 2024/11/23-21:06:21.820884 139957483730496 (Original Log Time 2024/11/23-21:06:21.820798) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 0 42 0 0] max score 0.01, MB/sec: 17.9 rd, 15.9 wr, level 4, files in(21, 54) out(42 +0 blob) MB in(825.9, 3431.0 +0.0 blob) out(3772.6 +0.0 blob), read-write-amplify(9.7) write-amplify(4.6) OK, records in: 79294926, records dropped: 5204243 output_compression: Snappy 2024/11/23-21:06:21.820887 139957483730496 (Original Log Time 2024/11/23-21:06:21.820850) EVENT_LOG_v1 {"time_micros": 1732367181820822, "cf": "default", "job": 35741, "event": "compaction_finished", "compaction_time_micros": 249037786, "compaction_time_cpu_micros": 97441979, "output_level": 4, "num_output_files": 42, "total_output_size": 3955807534, "num_input_records": 79294926, "num_output_records": 74090683, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 2581971820, "file_range_sync_nanos": 0, "file_fsync_nanos": 30326, "file_prepare_write_nanos": 5703704, "lsm_state": [0, 0, 0, 0, 42, 0, 0]} 2024/11/23-21:06:22.010298 139957483730496 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732366931,"executesMs":249870,"compactionJobId":35741,"attempt":0,"compactionInputRawBytes":10830461126,"compactionInputZipBytes":4463661836,"compactionOutputRawBytes":0,"compactionOutputZipBytes":3955807534,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-cf91804470c31df63b221a8e9bcc73dd-74cd76cb7e4f67ef-00"}}, response = 2024/11/23-21:06:22.010384 139957483730496 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35741/att-00: olev 4, work 249.871 s, result 7.741 ms, install 1.627 ms, input{raw 10.087 GiB zip 4.157 GiB}