2024/07/08-15:35:52.289295 139818989889088 DEBG dcompact_worker.cpp:2211: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00: fork to child time = 0.008930 sec 2024/07/08-15:35:52.289965 139818989889088 DEBG dcompact_worker.cpp:1011: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00 2024/07/08-15:35:52.315411 139818989889088 DEBG dcompact_worker.cpp:1031: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00 2024/07/08-15:35:52.317334 139818989889088 TRAC dcompact_worker.cpp:1081: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/07/08-15:35:52.317628 139818989889088 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/07/08-15:35:52.353722 139818989889088 [version_set.cc:6139] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-03530/att-00/MANIFEST-12625661 2024/07/08-15:35:52.418919 139818989889088 [version_set.cc:6182] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-03530/att-00/MANIFEST-12625661 succeeded,manifest_file_number is 12625661, next_file_number is 12628382, last_sequence is 147237045735, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 12628277 2024/07/08-15:35:52.418933 139818989889088 [version_set.cc:6197] Column family [default] (ID 0), log number is 0 2024/07/08-15:35:52.423210 139818989889088 [version_set.cc:5652] Creating manifest 12628382 2024/07/08-15:35:52.458388 139818989889088 DEBG dcompact_worker.cpp:1215: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00: bottommost_level: fake = 1, rpc = 0 2024/07/08-15:35:52.548920 139818989889088 [compaction_job.cc:2433] [default] [JOB 3530] Compacting 8@4 + 2508@5 files to L5, score -1.00, subcompactions 1 : 1 2024/07/08-15:35:52.549003 139818989889088 [compaction_job.cc:2441] [default]: Compaction start summary: Base version 2305 Base level 4, inputs: [12626359(172MB) 12626360(59MB) 12628316(18KB) 12626363(195MB) 12626534(194MB) 12626535(194MB) 12626536(171MB) 12626537(1865KB)], [12628379(330MB) 12628380(3286KB) 12620036(278MB) 12619621(206MB) 12628318(4108B) 12625776(25KB) 12545820(296MB) 12610894(291MB) 12545672(333MB) 12575517(386MB) 12575518(56MB) 12391997(386KB) 12392105(386KB) 12392104(15KB) 12392225(365KB) 12392226(105KB) 12392332(834KB) 12392583(1617KB) 12392707(459KB) 12392836(823KB) 12392935(828KB) 12393047(286KB) 12393162(70KB) 12393163(767KB) 12393429(816KB) 12393586(827KB) 12393735(834KB) 12393823(302KB) 12393830(211KB) 12394027(359KB) 12394032(118KB) 12394150(838KB) 12394241(844KB) 12394425(819KB) 12394530(460KB) 12394655(449KB) 12394738(265KB) 12394766(231KB) 12394905(831KB) 12394986(385KB) 12395002(27KB) 12395152(449KB) 12395254(435KB) 12395253(16KB) 12395359(305KB) 12395360(177KB) 12395457(450KB) 12395620(825KB) 12395777(793KB) 12395911(450KB) 12396032(843KB) 12396179(827KB) 12396319(447KB) 12396452(817KB) 12396551(424KB) 12396645(370KB) 12396644(117KB) 12396728(427KB) 12396726(4994B) 12396833(441KB) 12396933(284KB) 12396934(199KB) 12397052(841KB) 12397318(820KB) 12397397(440KB) 12397538(837KB) 12398283(195KB) 12398289(294KB) 12398404(417KB) 12398401(15KB) 12398499(259KB) 12398500(236KB) 12398603(444KB) 12398730(448KB) 12398831(443KB) 12398936(362KB) 12398934(111KB) 12399055(187KB) 12399056(298KB) 12399218(800KB) 12399355(416KB) 12399478(433KB) 12399568(435KB) 12399686(336KB) 12399685(154KB) 12399803(430KB) 12399940(437KB) 12400096(299KB) 12400118(191KB) 12400460(806KB) 12400775(426KB) 12401082(808KB) 12401382(429KB) 12401699(832KB) 12401943(439KB) 12402037(424KB) 12402140(419KB) 12402234(419KB) 12402325(399KB) 12402336(62KB) 12402425(422KB) 12402534(797KB) 12402650(266KB) 12402652(214KB) 12402765(801KB) 12402878(212KB) 12402879(267KB) 12403000(778KB) 12403130(49KB) 12403134(390KB) 12403217(373KB) 12403216(86KB) 12403299(396KB) 12403290(54KB) 12403394(359KB) 12403393(108KB) 12403495(217KB) 12403492(263KB) 12403595(254KB) 12403593(236KB) 12403688(102KB) 12403689(355KB) 12403798(314KB) 12403799(161KB) 12403910(414KB) 12403991(401KB) 12403993(49KB) 12404119(786KB) 12404230(157KB) 12404238(308KB) 12404319(417KB) 12404415(122KB) 12404416(344KB) 12404507(403KB) 12404506(5954B) 12404607(152KB)] 2024/07/08-15:35:52.554355 139818989889088 [top_zip_table.cc:59] core git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-15:35:52.554393 139818989889088 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-15:35:52.554430 139818989889088 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-15:35:52.554440 139818989889088 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 7f80399bfbd2071b6e0f5683e8ab7225b8e6e394 Author: leipeng Date: 2024-01-19 14:52:30 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/07/08-15:35:52.554450 139818989889088 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit 1240987ca13d0fe562c9b9c037155079f0877840 Author: leipeng Date: 2024-01-19 15:07:04 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/07/08-15:35:56.260827 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628383.sst) /dev/shm/ToplingTemp/Topling-21995-RAuKql 2024/07/08-15:35:58.606709 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628383.sst): first pass time = 2.35's, 228.863'MB/sec 2024/07/08-15:36:03.802983 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628383.sst): old prealloc_size = 590558003, real_size = 330792856 2024/07/08-15:36:03.803074 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628383.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.82 2024/07/08-15:36:05.973675 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628383.sst): second pass time = 5.19's, 95.088'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.17's, 159.447'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 9657.210'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.36's, 957.695'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 102552.467'MB/sec zip my value time = 5.19's, unzip length = 0.494'GB zip my value throughput = 95.088'MB/sec zip pipeline throughput = 95.088'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.82 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4981 dict = 1.00 all = 1.5512 } Zip/UnZip{ index = 0.0195 value = 0.6675 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 110.451097 GB avg = 0.177 KB (by entry num) total key len = 13.412704 GB avg = 0.022 KB total ukey len = 7.455210 GB avg = 0.010 KB total ukey num = 0.730994369 Billion total entry num = 0.622445815 Billion write speed all = 38.195970519 MB/sec (with seq num) write speed all = 36.660417953 MB/sec (without seq num) 2024/07/08-15:36:06.348469 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628383: 2684355 keys, 346109771 bytes, temperature: kWarm 2024/07/08-15:36:06.348553 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424166348507, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628383, "file_size": 346109771, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142187669095, "largest_seqno": 147192754356, "table_properties": {"data_size": 329701936, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720424156, "oldest_key_time": 0, "file_creation_time": 1720424156, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628383, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:36:06.450586 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628384.sst) /dev/shm/ToplingTemp/Topling-21995-Z8B08n 2024/07/08-15:36:13.618531 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628384.sst): first pass time = 7.17's, 74.899'MB/sec 2024/07/08-15:36:18.792382 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628384.sst): old prealloc_size = 590558003, real_size = 330816376 2024/07/08-15:36:18.792467 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628384.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2684355 prefix = 4 raw-key = 0.0107 GB zip-key = 0.0004 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.4939 GB zip-val = 0.3297 GB avg-val = 184.00 avg-zval = 122.83 2024/07/08-15:36:20.965171 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628384.sst): second pass time = 5.17's, 95.505'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 2.17's, 159.304'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 13672.737'MB/sec (index lex order gen) rebuild zvType time = 1.81's, 0.371'MB/sec write SST data time = 0.36's, 949.942'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 4.914'MB, throughput = 97888.484'MB/sec zip my value time = 5.17's, unzip length = 0.494'GB zip my value throughput = 95.505'MB/sec zip pipeline throughput = 95.505'MB/sec entries = 2684355 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 122.83 usrkeys = 2684355 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 122.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2684355 bytes = 11072976 } seq expand size = 18790485 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0215 GB value = 0.4939 GB dict = 4.91 MB all = 0.5369 GB } __ZipSize{ index = 0.0004 GB value = 0.3297 GB dict = 4.91 MB all = 0.3461 GB } UnZip/Zip{ index = 51.1657 value = 1.4980 dict = 1.00 all = 1.5511 } Zip/UnZip{ index = 0.0195 value = 0.6676 dict = 1.00 all = 0.6447 } ---------------------------- total value len = 110.963808 GB avg = 0.178 KB (by entry num) total key len = 13.455653 GB avg = 0.022 KB total ukey len = 7.476684 GB avg = 0.010 KB total ukey num = 0.733678724 Billion total entry num = 0.625130170 Billion write speed all = 38.190766685 MB/sec (with seq num) write speed all = 36.655688480 MB/sec (without seq num) 2024/07/08-15:36:21.341456 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628384: 2684355 keys, 346133259 bytes, temperature: kWarm 2024/07/08-15:36:21.341525 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424181341490, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628384, "file_size": 346133259, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142531386549, "largest_seqno": 147192754443, "table_properties": {"data_size": 329725456, "index_size": 419712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 42949680, "raw_average_key_size": 16, "raw_value_size": 493921320, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 2684355, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720424166, "oldest_key_time": 0, "file_creation_time": 1720424166, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628384, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:36:21.443821 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628385.sst) /dev/shm/ToplingTemp/Topling-21995-QebwTm 2024/07/08-15:36:23.319428 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628385.sst): first pass time = 1.88's, 128.974'MB/sec 2024/07/08-15:36:25.710102 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628385.sst): old prealloc_size = 590558003, real_size = 157512144 2024/07/08-15:36:25.710174 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628385.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 1209495 prefix = 4 raw-key = 0.0048 GB zip-key = 0.0002 GB avg-key = 4.00 avg-zkey = 0.16 raw-val = 0.2225 GB zip-val = 0.1570 GB avg-val = 184.00 avg-zval = 129.82 2024/07/08-15:36:26.775726 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628385.sst): second pass time = 2.39's, 93.142'MB/sec, value only(92.0% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 154.288'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 6331.694'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.340'MB/sec write SST data time = 0.18's, 937.216'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 2.211'MB, throughput = 53505.093'MB/sec zip my value time = 2.39's, unzip length = 0.223'GB zip my value throughput = 93.142'MB/sec zip pipeline throughput = 93.142'MB/sec entries = 1209495 avg-key = 16.00 avg-zkey = 0.16 avg-val = 184.00 avg-zval = 129.82 usrkeys = 1209495 avg-key = 8.00 avg-zkey = 0.16 avg-val = 191.00 avg-zval = 129.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 1209495 bytes = 4989184 } seq expand size = 8466465 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0097 GB value = 0.2225 GB dict = 2.21 MB all = 0.2419 GB } __ZipSize{ index = 0.0002 GB value = 0.1570 GB dict = 2.21 MB all = 0.1644 GB } UnZip/Zip{ index = 51.1285 value = 1.4173 dict = 1.00 all = 1.4713 } Zip/UnZip{ index = 0.0196 value = 0.7056 dict = 1.00 all = 0.6797 } ---------------------------- total value len = 111.194822 GB avg = 0.178 KB (by entry num) total key len = 13.475005 GB avg = 0.022 KB total ukey len = 7.486360 GB avg = 0.010 KB total ukey num = 0.734888219 Billion total entry num = 0.626339665 Billion write speed all = 38.199485710 MB/sec (with seq num) write speed all = 36.664175770 MB/sec (without seq num) 2024/07/08-15:36:26.979285 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628385: 1209495 keys, 164411333 bytes, temperature: kWarm 2024/07/08-15:36:26.979353 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424186979320, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628385, "file_size": 164411333, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142412098167, "largest_seqno": 147192754435, "table_properties": {"data_size": 157020368, "index_size": 189248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 19351920, "raw_average_key_size": 16, "raw_value_size": 222547080, "raw_average_value_size": 184, "num_data_blocks": 1, "num_entries": 1209495, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 184, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1720424181, "oldest_key_time": 0, "file_creation_time": 1720424181, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628385, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:36:27.072479 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628386.sst) /dev/shm/ToplingTemp/Topling-21995-iyCcAo 2024/07/08-15:36:27.086807 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628386.sst): first pass time = 0.01's, 0.178'MB/sec 2024/07/08-15:36:27.173708 139818989889088 [top_zip_table_builder.cc:2718] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628386.sst): part-0: reject ZipStore ratio = 0.907 2024/07/08-15:36:27.174179 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628386.sst): old prealloc_size = 590558003, real_size = 3008 2024/07/08-15:36:27.174225 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628386.sst): index type = UintIndex_IL_256_32 , store type = ZipOffsetBlobStore usrkeys = 19 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 4.00 avg-zkey = 14.32 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 117.95 avg-zval = 133.05 2024/07/08-15:36:27.174693 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628386.sst): second pass time = 0.09's, 0.026'MB/sec, value only(88.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 8.121'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 12.363'MB/sec (index lex order gen) rebuild zvType time = 0.00's, 0.027'MB/sec write SST data time = 0.00's, 13.546'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 0.000'MB/sec zip my value time = 0.09's, unzip length = 0.000'GB zip my value throughput = 0.026'MB/sec zip pipeline throughput = 240.677'MB/sec entries = 19 avg-key = 16.00 avg-zkey = 16.84 avg-val = 117.95 avg-zval = 133.05 usrkeys = 19 avg-key = 8.00 avg-zkey = 16.84 avg-val = 124.95 avg-zval = 133.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 19 bytes = 96 } seq expand size = 133 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } __ZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } UnZip/Zip{ index = 0.4750 value = 0.8865 dict = 0.00 all = 0.6195 } Zip/UnZip{ index = 2.1053 value = 1.1281 dict = 0.00 all = 1.6141 } ---------------------------- total value len = 111.194824 GB avg = 0.178 KB (by entry num) total key len = 13.475005 GB avg = 0.022 KB total ukey len = 7.486361 GB avg = 0.010 KB total ukey num = 0.734888238 Billion total entry num = 0.626339684 Billion write speed all = 38.194817336 MB/sec (with seq num) write speed all = 36.659695013 MB/sec (without seq num) 2024/07/08-15:36:27.180728 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628386: 19 keys, 4108 bytes, temperature: kWarm 2024/07/08-15:36:27.180802 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424187180758, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628386, "file_size": 4108, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142422558047, "largest_seqno": 147235923177, "table_properties": {"data_size": 2528, "index_size": 320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 304, "raw_average_key_size": 16, "raw_value_size": 2241, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 19, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 16, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-ZipOffsetBlobStore", "creation_time": 1720424187, "oldest_key_time": 0, "file_creation_time": 1720424187, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628386, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:36:27.194311 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628387.sst) /dev/shm/ToplingTemp/Topling-21995-Yod75m 2024/07/08-15:36:31.309614 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628387.sst): first pass time = 4.12's, 0.006'MB/sec 2024/07/08-15:36:31.392033 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628387.sst): old prealloc_size = 590558003, real_size = 27488 2024/07/08-15:36:31.392077 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628387.sst): index type = FixedLenKeyIndex , store type = DictZipBlobStore usrkeys = 190 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 50.53 raw-val = 0.0000 GB zip-val = 0.0000 GB avg-val = 120.69 avg-zval = 93.05 2024/07/08-15:36:31.392604 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628387.sst): second pass time = 0.08's, 0.281'MB/sec, value only(85.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 51.303'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 441.582'MB/sec (index lex order gen) rebuild zvType time = 0.00's, 0.193'MB/sec write SST data time = 0.00's, 94.950'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.000'MB, throughput = 16.854'MB/sec zip my value time = 0.08's, unzip length = 0.000'GB zip my value throughput = 0.281'MB/sec zip pipeline throughput = 273.906'MB/sec entries = 190 avg-key = 20.00 avg-zkey = 50.53 avg-val = 120.69 avg-zval = 93.05 usrkeys = 190 avg-key = 12.00 avg-zkey = 50.53 avg-val = 127.69 avg-zval = 93.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 190 bytes = 800 } seq expand size = 1330 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } __ZipSize{ index = 0.0000 GB value = 0.0000 GB dict = 0.00 MB all = 0.0000 GB } UnZip/Zip{ index = 0.2375 value = 1.2971 dict = 1.00 all = 0.8991 } Zip/UnZip{ index = 4.2105 value = 0.7710 dict = 1.00 all = 1.1122 } ---------------------------- total value len = 111.194849 GB avg = 0.178 KB (by entry num) total key len = 13.475009 GB avg = 0.022 KB total ukey len = 7.486363 GB avg = 0.010 KB total ukey num = 0.734888428 Billion total entry num = 0.626339874 Billion write speed all = 38.145532871 MB/sec (with seq num) write speed all = 36.612391264 MB/sec (without seq num) 2024/07/08-15:36:31.397935 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628387: 190 keys, 29732 bytes, temperature: kWarm 2024/07/08-15:36:31.397993 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424191397964, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628387, "file_size": 29732, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142422546641, "largest_seqno": 147235923176, "table_properties": {"data_size": 17680, "index_size": 9600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3800, "raw_average_key_size": 20, "raw_value_size": 22932, "raw_average_value_size": 120, "num_data_blocks": 1, "num_entries": 190, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-FixedLenKeyIndex-DictZipBlobStore", "creation_time": 1720424187, "oldest_key_time": 0, "file_creation_time": 1720424187, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628387, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:36:31.413908 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628388.sst) /dev/shm/ToplingTemp/Topling-21995-5rKbso 2024/07/08-15:36:38.381026 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628388.sst): first pass time = 6.97's, 77.058'MB/sec 2024/07/08-15:36:54.316276 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628388.sst): old prealloc_size = 590558003, real_size = 398305872 2024/07/08-15:36:54.316755 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628388.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 790943 prefix = 4 raw-key = 0.0095 GB zip-key = 0.0013 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.5179 GB zip-val = 0.3968 GB avg-val = 654.77 avg-zval = 501.64 2024/07/08-15:36:54.317570 139818989889088 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628388.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0020 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/07/08-15:36:55.431607 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628388.sst): second pass time = 15.92's, 32.524'MB/sec, value only(96.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 364.504'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3088.983'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.233'MB/sec write SST data time = 0.26's, 1534.243'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.167'MB, throughput = 107402.162'MB/sec zip my value time = 15.92's, unzip length = 0.518'GB zip my value throughput = 32.524'MB/sec zip pipeline throughput = 58.418'MB/sec entries = 790943 avg-key = 24.00 avg-zkey = 1.70 avg-val = 654.77 avg-zval = 501.64 usrkeys = 790943 avg-key = 16.00 avg-zkey = 1.70 avg-val = 661.77 avg-zval = 501.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 790943 bytes = 3262656 } seq expand size = 5536601 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0127 GB value = 0.5179 GB dict = 5.17 MB all = 0.5369 GB } __ZipSize{ index = 0.0013 GB value = 0.3968 GB dict = 5.17 MB all = 0.4065 GB } UnZip/Zip{ index = 9.4326 value = 1.3053 dict = 1.00 all = 1.3206 } Zip/UnZip{ index = 0.1060 value = 0.7661 dict = 1.00 all = 0.7572 } ---------------------------- total value len = 111.718274 GB avg = 0.178 KB (by entry num) total key len = 13.493992 GB avg = 0.022 KB total ukey len = 7.499018 GB avg = 0.010 KB total ukey num = 0.735679371 Billion total entry num = 0.627130817 Billion write speed all = 38.031761043 MB/sec (with seq num) write speed all = 36.507891847 MB/sec (without seq num) 2024/07/08-15:36:55.881432 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628388: 790943 keys, 406544901 bytes, temperature: kWarm 2024/07/08-15:36:55.881504 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424215881468, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628388, "file_size": 406544901, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142138602368, "largest_seqno": 147192754458, "table_properties": {"data_size": 396766336, "index_size": 1341632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 18982632, "raw_average_key_size": 24, "raw_value_size": 517888973, "raw_average_value_size": 654, "num_data_blocks": 1, "num_entries": 790943, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720424191, "oldest_key_time": 0, "file_creation_time": 1720424191, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628388, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:36:56.006800 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628389.sst) /dev/shm/ToplingTemp/Topling-21995-JTZ4wo 2024/07/08-15:36:59.774868 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628389.sst): first pass time = 3.77's, 142.481'MB/sec 2024/07/08-15:37:11.645351 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628389.sst): old prealloc_size = 590558003, real_size = 397327472 2024/07/08-15:37:11.645805 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628389.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 790915 prefix = 4 raw-key = 0.0095 GB zip-key = 0.0013 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.5179 GB zip-val = 0.3958 GB avg-val = 654.80 avg-zval = 500.42 2024/07/08-15:37:11.646600 139818989889088 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628389.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0020 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/07/08-15:37:12.730131 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628389.sst): second pass time = 11.87's, 43.635'MB/sec, value only(96.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 373.879'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3312.768'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.243'MB/sec write SST data time = 0.27's, 1496.835'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.176'MB, throughput = 112733.762'MB/sec zip my value time = 11.87's, unzip length = 0.518'GB zip my value throughput = 43.635'MB/sec zip pipeline throughput = 43.635'MB/sec entries = 790915 avg-key = 24.00 avg-zkey = 1.70 avg-val = 654.80 avg-zval = 500.42 usrkeys = 790915 avg-key = 16.00 avg-zkey = 1.70 avg-val = 661.80 avg-zval = 500.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 790915 bytes = 3262544 } seq expand size = 5536405 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0127 GB value = 0.5179 GB dict = 5.18 MB all = 0.5369 GB } __ZipSize{ index = 0.0013 GB value = 0.3958 GB dict = 5.18 MB all = 0.4056 GB } UnZip/Zip{ index = 9.4368 value = 1.3085 dict = 1.00 all = 1.3237 } Zip/UnZip{ index = 0.1060 value = 0.7642 dict = 1.00 all = 0.7554 } ---------------------------- total value len = 112.241700 GB avg = 0.179 KB (by entry num) total key len = 13.512974 GB avg = 0.022 KB total ukey len = 7.511673 GB avg = 0.010 KB total ukey num = 0.736470286 Billion total entry num = 0.627921732 Billion write speed all = 37.996867026 MB/sec (with seq num) write speed all = 36.479050925 MB/sec (without seq num) 2024/07/08-15:37:13.171958 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628389: 790915 keys, 405575431 bytes, temperature: kWarm 2024/07/08-15:37:13.172029 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424233171995, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628389, "file_size": 405575431, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142138586461, "largest_seqno": 147192754025, "table_properties": {"data_size": 395788576, "index_size": 1340992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 18981960, "raw_average_key_size": 24, "raw_value_size": 517889281, "raw_average_value_size": 654, "num_data_blocks": 1, "num_entries": 790915, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720424215, "oldest_key_time": 0, "file_creation_time": 1720424215, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628389, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:37:13.295360 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628390.sst) /dev/shm/ToplingTemp/Topling-21995-PsGi8l 2024/07/08-15:37:18.994801 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628390.sst): first pass time = 5.70's, 94.198'MB/sec 2024/07/08-15:37:30.849830 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628390.sst): old prealloc_size = 590558003, real_size = 397496336 2024/07/08-15:37:30.850352 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628390.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 791050 prefix = 4 raw-key = 0.0095 GB zip-key = 0.0013 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.5179 GB zip-val = 0.3960 GB avg-val = 654.68 avg-zval = 500.55 2024/07/08-15:37:30.851109 139818989889088 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628390.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0020 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/07/08-15:37:31.927177 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628390.sst): second pass time = 11.85's, 43.693'MB/sec, value only(96.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 376.616'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2873.892'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.244'MB/sec write SST data time = 0.27's, 1529.325'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.174'MB, throughput = 110399.535'MB/sec zip my value time = 11.85's, unzip length = 0.518'GB zip my value throughput = 43.693'MB/sec zip pipeline throughput = 43.693'MB/sec entries = 791050 avg-key = 24.00 avg-zkey = 1.70 avg-val = 654.68 avg-zval = 500.55 usrkeys = 791050 avg-key = 16.00 avg-zkey = 1.70 avg-val = 661.68 avg-zval = 500.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 791050 bytes = 3263104 } seq expand size = 5537350 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0127 GB value = 0.5179 GB dict = 5.17 MB all = 0.5369 GB } __ZipSize{ index = 0.0013 GB value = 0.3960 GB dict = 5.17 MB all = 0.4057 GB } UnZip/Zip{ index = 9.4298 value = 1.3079 dict = 1.00 all = 1.3232 } Zip/UnZip{ index = 0.1060 value = 0.7646 dict = 1.00 all = 0.7558 } ---------------------------- total value len = 112.765123 GB avg = 0.179 KB (by entry num) total key len = 13.531959 GB avg = 0.022 KB total ukey len = 7.524329 GB avg = 0.010 KB total ukey num = 0.737261336 Billion total entry num = 0.628712782 Billion write speed all = 37.940685020 MB/sec (with seq num) write speed all = 36.429720984 MB/sec (without seq num) 2024/07/08-15:37:32.370975 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628390: 791050 keys, 405742775 bytes, temperature: kWarm 2024/07/08-15:37:32.371048 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424252371012, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628390, "file_size": 405742775, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142138598954, "largest_seqno": 147192749393, "table_properties": {"data_size": 395956160, "index_size": 1342208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 18985200, "raw_average_key_size": 24, "raw_value_size": 517886076, "raw_average_value_size": 654, "num_data_blocks": 1, "num_entries": 791050, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720424233, "oldest_key_time": 0, "file_creation_time": 1720424233, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628390, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:37:32.492544 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628391.sst) /dev/shm/ToplingTemp/Topling-21995-s9t2Bo 2024/07/08-15:37:35.121779 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628391.sst): first pass time = 2.63's, 204.197'MB/sec 2024/07/08-15:37:46.794360 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628391.sst): old prealloc_size = 590558003, real_size = 397492816 2024/07/08-15:37:46.794830 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628391.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 790739 prefix = 4 raw-key = 0.0095 GB zip-key = 0.0013 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.5179 GB zip-val = 0.3960 GB avg-val = 654.95 avg-zval = 500.74 2024/07/08-15:37:46.795615 139818989889088 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628391.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0020 GB, waited 0.000 sec, Key+Value = 0.537 GB 2024/07/08-15:37:47.865377 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628391.sst): second pass time = 11.67's, 44.377'MB/sec, value only(96.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 378.846'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3208.949'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.245'MB/sec write SST data time = 0.26's, 1536.569'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 5.181'MB, throughput = 112257.427'MB/sec zip my value time = 11.67's, unzip length = 0.518'GB zip my value throughput = 44.377'MB/sec zip pipeline throughput = 44.377'MB/sec entries = 790739 avg-key = 24.00 avg-zkey = 1.70 avg-val = 654.95 avg-zval = 500.74 usrkeys = 790739 avg-key = 16.00 avg-zkey = 1.70 avg-val = 661.95 avg-zval = 500.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 790739 bytes = 3261808 } seq expand size = 5535173 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0127 GB value = 0.5179 GB dict = 5.18 MB all = 0.5369 GB } __ZipSize{ index = 0.0013 GB value = 0.3960 GB dict = 5.18 MB all = 0.4057 GB } UnZip/Zip{ index = 9.4320 value = 1.3080 dict = 1.00 all = 1.3232 } Zip/UnZip{ index = 0.1060 value = 0.7645 dict = 1.00 all = 0.7558 } ---------------------------- total value len = 113.288552 GB avg = 0.180 KB (by entry num) total key len = 13.550937 GB avg = 0.022 KB total ukey len = 7.536981 GB avg = 0.010 KB total ukey num = 0.738052075 Billion total entry num = 0.629503521 Billion write speed all = 37.922058817 MB/sec (with seq num) write speed all = 36.416403451 MB/sec (without seq num) 2024/07/08-15:37:48.309470 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628391: 790739 keys, 405745832 bytes, temperature: kWarm 2024/07/08-15:37:48.309540 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424268309507, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628391, "file_size": 405745832, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142138607838, "largest_seqno": 147192754432, "table_properties": {"data_size": 395953600, "index_size": 1341376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 18977736, "raw_average_key_size": 24, "raw_value_size": 517893339, "raw_average_value_size": 654, "num_data_blocks": 1, "num_entries": 790739, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720424252, "oldest_key_time": 0, "file_creation_time": 1720424252, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628391, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:37:48.421465 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628392.sst) /dev/shm/ToplingTemp/Topling-21995-rI5C8j 2024/07/08-15:37:49.196225 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628392.sst): first pass time = 0.77's, 163.695'MB/sec 2024/07/08-15:37:50.790302 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628392.sst): old prealloc_size = 590558003, real_size = 95386368 2024/07/08-15:37:50.790465 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628392.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 186837 prefix = 4 raw-key = 0.0022 GB zip-key = 0.0003 GB avg-key = 12.00 avg-zkey = 1.70 raw-val = 0.1223 GB zip-val = 0.0950 GB avg-val = 654.75 avg-zval = 508.58 2024/07/08-15:37:50.790682 139818989889088 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628392.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.127 GB 2024/07/08-15:37:51.050476 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628392.sst): second pass time = 1.59's, 76.779'MB/sec, value only(96.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.26's, 374.337'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2440.846'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.241'MB/sec write SST data time = 0.07's, 1467.319'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.280'MB, throughput = 29575.018'MB/sec zip my value time = 1.59's, unzip length = 0.122'GB zip my value throughput = 76.779'MB/sec zip pipeline throughput = 76.779'MB/sec entries = 186837 avg-key = 24.00 avg-zkey = 1.70 avg-val = 654.75 avg-zval = 508.58 usrkeys = 186837 avg-key = 16.00 avg-zkey = 1.70 avg-val = 661.75 avg-zval = 508.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 186837 bytes = 770720 } seq expand size = 1307859 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0030 GB value = 0.1223 GB dict = 1.28 MB all = 0.1268 GB } __ZipSize{ index = 0.0003 GB value = 0.0950 GB dict = 1.28 MB all = 0.0974 GB } UnZip/Zip{ index = 9.3850 value = 1.2874 dict = 1.00 all = 1.3021 } Zip/UnZip{ index = 0.1066 value = 0.7767 dict = 1.00 all = 0.7680 } ---------------------------- total value len = 113.412192 GB avg = 0.180 KB (by entry num) total key len = 13.555421 GB avg = 0.022 KB total ukey len = 7.539971 GB avg = 0.010 KB total ukey num = 0.738238912 Billion total entry num = 0.629690358 Billion write speed all = 37.924250647 MB/sec (with seq num) write speed all = 36.419581258 MB/sec (without seq num) 2024/07/08-15:37:51.159993 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628392: 186837 keys, 97392600 bytes, temperature: kWarm 2024/07/08-15:37:51.160062 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424271160028, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628392, "file_size": 97392600, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 142138677206, "largest_seqno": 147192751977, "table_properties": {"data_size": 95020976, "index_size": 318528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4484088, "raw_average_key_size": 24, "raw_value_size": 122331946, "raw_average_value_size": 654, "num_data_blocks": 1, "num_entries": 186837, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 24, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720424268, "oldest_key_time": 0, "file_creation_time": 1720424268, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628392, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:37:51.197272 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628393.sst) /dev/shm/ToplingTemp/Topling-21995-ZF8WXl 2024/07/08-15:37:58.479171 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628393.sst): first pass time = 7.28's, 73.727'MB/sec 2024/07/08-15:38:03.938588 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628393.sst): old prealloc_size = 590558003, real_size = 130401096 2024/07/08-15:38:03.938662 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628393.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 7386600 prefix = 4 raw-key = 0.0591 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.3891 GB zip-val = 0.1286 GB avg-val = 52.68 avg-zval = 17.40 2024/07/08-15:38:04.595675 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628393.sst): second pass time = 5.46's, 71.302'MB/sec, value only(72.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.66's, 201.578'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 8.784'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 18.471'MB/sec write SST data time = 0.56's, 237.760'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 3.897'MB, throughput = 86550.953'MB/sec zip my value time = 5.46's, unzip length = 0.389'GB zip my value throughput = 71.302'MB/sec zip pipeline throughput = 71.302'MB/sec entries = 7386600 avg-key = 20.00 avg-zkey = 0.00 avg-val = 52.68 avg-zval = 17.40 usrkeys = 7386600 avg-key = 12.00 avg-zkey = 0.00 avg-val = 52.68 avg-zval = 17.40 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 4 UnZipSize{ index = 0.0886 GB value = 0.3891 GB dict = 3.90 MB all = 0.5369 GB } __ZipSize{ index = 0.0000 GB value = 0.1286 GB dict = 3.90 MB all = 0.1325 GB } UnZip/Zip{ index =461662.5000 value = 3.0270 dict = 1.00 all = 4.0533 } Zip/UnZip{ index = 0.0000 value = 0.3304 dict = 1.00 all = 0.2467 } ---------------------------- total value len = 113.801331 GB avg = 0.179 KB (by entry num) total key len = 13.703153 GB avg = 0.022 KB total ukey len = 7.628610 GB avg = 0.010 KB total ukey num = 0.745625512 Billion total entry num = 0.637076958 Billion write speed all = 37.931146311 MB/sec (with seq num) write speed all = 36.414960611 MB/sec (without seq num) 2024/07/08-15:38:04.741015 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628393: 7386600 keys, 132452146 bytes, temperature: kWarm 2024/07/08-15:38:04.741102 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424284741057, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628393, "file_size": 132452146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 128554064, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 147732000, "raw_average_key_size": 20, "raw_value_size": 389138970, "raw_average_value_size": 52, "num_data_blocks": 1, "num_entries": 7386600, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1720424271, "oldest_key_time": 0, "file_creation_time": 1720424271, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628393, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:38:04.836703 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628394.sst) /dev/shm/ToplingTemp/Topling-21995-3vrItk 2024/07/08-15:38:11.920138 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628394.sst): first pass time = 7.08's, 75.793'MB/sec 2024/07/08-15:38:17.322757 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628394.sst): old prealloc_size = 590558003, real_size = 128469824 2024/07/08-15:38:17.322833 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628394.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 7377535 prefix = 4 raw-key = 0.0590 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.3893 GB zip-val = 0.1266 GB avg-val = 52.77 avg-zval = 17.16 2024/07/08-15:38:17.970111 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628394.sst): second pass time = 5.40's, 72.111'MB/sec, value only(72.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 201.632'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 8.326'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 19.383'MB/sec write SST data time = 0.55's, 236.387'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 3.898'MB, throughput = 92874.092'MB/sec zip my value time = 5.40's, unzip length = 0.389'GB zip my value throughput = 72.111'MB/sec zip pipeline throughput = 72.111'MB/sec entries = 7377535 avg-key = 20.00 avg-zkey = 0.00 avg-val = 52.77 avg-zval = 17.16 usrkeys = 7377535 avg-key = 12.00 avg-zkey = 0.00 avg-val = 52.77 avg-zval = 17.16 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 4 UnZipSize{ index = 0.0885 GB value = 0.3893 GB dict = 3.90 MB all = 0.5369 GB } __ZipSize{ index = 0.0000 GB value = 0.1266 GB dict = 3.90 MB all = 0.1305 GB } UnZip/Zip{ index =461095.9375 value = 3.0746 dict = 1.00 all = 4.1132 } Zip/UnZip{ index = 0.0000 value = 0.3252 dict = 1.00 all = 0.2431 } ---------------------------- total value len = 114.190651 GB avg = 0.177 KB (by entry num) total key len = 13.850704 GB avg = 0.021 KB total ukey len = 7.717140 GB avg = 0.010 KB total ukey num = 0.753003047 Billion total entry num = 0.644454493 Billion write speed all = 37.939906188 MB/sec (with seq num) write speed all = 36.412240809 MB/sec (without seq num) 2024/07/08-15:38:18.111763 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628394: 7377535 keys, 130524879 bytes, temperature: kWarm 2024/07/08-15:38:18.111835 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424298111800, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628394, "file_size": 130524879, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 126625104, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 147550700, "raw_average_key_size": 20, "raw_value_size": 389320250, "raw_average_value_size": 52, "num_data_blocks": 1, "num_entries": 7377535, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1720424284, "oldest_key_time": 0, "file_creation_time": 1720424284, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628394, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:38:18.205833 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628395.sst) /dev/shm/ToplingTemp/Topling-21995-1t59Ym 2024/07/08-15:38:26.277921 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628395.sst): first pass time = 8.07's, 66.510'MB/sec 2024/07/08-15:38:31.759710 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628395.sst): old prealloc_size = 590558003, real_size = 128974920 2024/07/08-15:38:31.759781 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628395.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 7376801 prefix = 4 raw-key = 0.0590 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.3893 GB zip-val = 0.1271 GB avg-val = 52.78 avg-zval = 17.23 2024/07/08-15:38:32.410572 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628395.sst): second pass time = 5.48's, 71.054'MB/sec, value only(72.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 201.321'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 9.112'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 19.190'MB/sec write SST data time = 0.55's, 236.207'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 3.898'MB, throughput = 66553.220'MB/sec zip my value time = 5.48's, unzip length = 0.389'GB zip my value throughput = 71.054'MB/sec zip pipeline throughput = 71.054'MB/sec entries = 7376801 avg-key = 20.00 avg-zkey = 0.00 avg-val = 52.78 avg-zval = 17.23 usrkeys = 7376801 avg-key = 12.00 avg-zkey = 0.00 avg-val = 52.78 avg-zval = 17.23 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 4 UnZipSize{ index = 0.0885 GB value = 0.3893 GB dict = 3.90 MB all = 0.5369 GB } __ZipSize{ index = 0.0000 GB value = 0.1271 GB dict = 3.90 MB all = 0.1310 GB } UnZip/Zip{ index =461050.0625 value = 3.0625 dict = 1.00 all = 4.0973 } Zip/UnZip{ index = 0.0000 value = 0.3265 dict = 1.00 all = 0.2441 } ---------------------------- total value len = 114.579986 GB avg = 0.176 KB (by entry num) total key len = 13.998240 GB avg = 0.021 KB total ukey len = 7.805662 GB avg = 0.010 KB total ukey num = 0.760379848 Billion total entry num = 0.651831294 Billion write speed all = 37.936661055 MB/sec (with seq num) write speed all = 36.398092433 MB/sec (without seq num) 2024/07/08-15:38:32.568969 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628395: 7376801 keys, 131029658 bytes, temperature: kWarm 2024/07/08-15:38:32.569039 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424312569005, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628395, "file_size": 131029658, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 127130384, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 147536020, "raw_average_key_size": 20, "raw_value_size": 389334952, "raw_average_value_size": 52, "num_data_blocks": 1, "num_entries": 7376801, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1720424298, "oldest_key_time": 0, "file_creation_time": 1720424298, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628395, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:38:32.648781 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628396.sst) /dev/shm/ToplingTemp/Topling-21995-m3Jxvo 2024/07/08-15:38:41.161391 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628396.sst): first pass time = 8.51's, 63.068'MB/sec 2024/07/08-15:38:46.626662 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628396.sst): old prealloc_size = 590558003, real_size = 129147168 2024/07/08-15:38:46.626738 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628396.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 7376670 prefix = 4 raw-key = 0.0590 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.3893 GB zip-val = 0.1273 GB avg-val = 52.78 avg-zval = 17.26 2024/07/08-15:38:47.273292 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628396.sst): second pass time = 5.46's, 71.268'MB/sec, value only(72.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 202.909'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 8.174'MB/sec (index lex order gen) rebuild zvType time = 0.09's, 19.493'MB/sec write SST data time = 0.55's, 237.695'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 3.899'MB, throughput = 90672.938'MB/sec zip my value time = 5.46's, unzip length = 0.389'GB zip my value throughput = 71.268'MB/sec zip pipeline throughput = 71.268'MB/sec entries = 7376670 avg-key = 20.00 avg-zkey = 0.00 avg-val = 52.78 avg-zval = 17.26 usrkeys = 7376670 avg-key = 12.00 avg-zkey = 0.00 avg-val = 52.78 avg-zval = 17.26 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 4 UnZipSize{ index = 0.0885 GB value = 0.3893 GB dict = 3.90 MB all = 0.5369 GB } __ZipSize{ index = 0.0000 GB value = 0.1273 GB dict = 3.90 MB all = 0.1312 GB } UnZip/Zip{ index =461041.8750 value = 3.0584 dict = 1.00 all = 4.0919 } Zip/UnZip{ index = 0.0000 value = 0.3270 dict = 1.00 all = 0.2444 } ---------------------------- total value len = 114.969323 GB avg = 0.174 KB (by entry num) total key len = 14.145773 GB avg = 0.021 KB total ukey len = 7.894182 GB avg = 0.010 KB total ukey num = 0.767756518 Billion total entry num = 0.659207964 Billion write speed all = 37.928738155 MB/sec (with seq num) write speed all = 36.379551299 MB/sec (without seq num) 2024/07/08-15:38:47.416360 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628396: 7376670 keys, 131202748 bytes, temperature: kWarm 2024/07/08-15:38:47.416443 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424327416401, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628396, "file_size": 131202748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 127302640, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 147533400, "raw_average_key_size": 20, "raw_value_size": 389337534, "raw_average_value_size": 52, "num_data_blocks": 1, "num_entries": 7376670, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1720424312, "oldest_key_time": 0, "file_creation_time": 1720424312, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628396, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:38:47.497364 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628397.sst) /dev/shm/ToplingTemp/Topling-21995-4yA1xo 2024/07/08-15:38:55.947781 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628397.sst): first pass time = 8.45's, 63.532'MB/sec 2024/07/08-15:39:01.391290 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628397.sst): old prealloc_size = 590558003, real_size = 129471080 2024/07/08-15:39:01.391361 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628397.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 7376627 prefix = 4 raw-key = 0.0590 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.3893 GB zip-val = 0.1276 GB avg-val = 52.78 avg-zval = 17.30 2024/07/08-15:39:02.039924 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628397.sst): second pass time = 5.44's, 71.548'MB/sec, value only(72.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 202.774'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 8.649'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 19.230'MB/sec write SST data time = 0.55's, 237.965'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 3.899'MB, throughput = 84486.260'MB/sec zip my value time = 5.44's, unzip length = 0.389'GB zip my value throughput = 71.548'MB/sec zip pipeline throughput = 71.548'MB/sec entries = 7376627 avg-key = 20.00 avg-zkey = 0.00 avg-val = 52.78 avg-zval = 17.30 usrkeys = 7376627 avg-key = 12.00 avg-zkey = 0.00 avg-val = 52.78 avg-zval = 17.30 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 4 UnZipSize{ index = 0.0885 GB value = 0.3893 GB dict = 3.90 MB all = 0.5369 GB } __ZipSize{ index = 0.0000 GB value = 0.1276 GB dict = 3.90 MB all = 0.1315 GB } UnZip/Zip{ index =461039.1875 value = 3.0506 dict = 1.00 all = 4.0818 } Zip/UnZip{ index = 0.0000 value = 0.3278 dict = 1.00 all = 0.2450 } ---------------------------- total value len = 115.358662 GB avg = 0.173 KB (by entry num) total key len = 14.293306 GB avg = 0.021 KB total ukey len = 7.982701 GB avg = 0.010 KB total ukey num = 0.775133145 Billion total entry num = 0.666584591 Billion write speed all = 37.921949674 MB/sec (with seq num) write speed all = 36.362193174 MB/sec (without seq num) 2024/07/08-15:39:02.185132 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628397: 7376627 keys, 131526431 bytes, temperature: kWarm 2024/07/08-15:39:02.185221 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424342185169, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628397, "file_size": 131526431, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 127626544, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 147532540, "raw_average_key_size": 20, "raw_value_size": 389338431, "raw_average_value_size": 52, "num_data_blocks": 1, "num_entries": 7376627, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1720424327, "oldest_key_time": 0, "file_creation_time": 1720424327, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628397, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:39:02.268767 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628398.sst) /dev/shm/ToplingTemp/Topling-21995-jQeTFl 2024/07/08-15:39:10.896020 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628398.sst): first pass time = 8.63's, 62.230'MB/sec 2024/07/08-15:39:16.404896 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628398.sst): old prealloc_size = 590558003, real_size = 129695192 2024/07/08-15:39:16.404970 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628398.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 7376305 prefix = 4 raw-key = 0.0590 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.3893 GB zip-val = 0.1279 GB avg-val = 52.78 avg-zval = 17.33 2024/07/08-15:39:17.059900 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628398.sst): second pass time = 5.51's, 70.706'MB/sec, value only(72.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 201.148'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 9.096'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 18.142'MB/sec write SST data time = 0.55's, 238.106'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 3.898'MB, throughput = 82165.929'MB/sec zip my value time = 5.51's, unzip length = 0.389'GB zip my value throughput = 70.706'MB/sec zip pipeline throughput = 70.706'MB/sec entries = 7376305 avg-key = 20.00 avg-zkey = 0.00 avg-val = 52.78 avg-zval = 17.33 usrkeys = 7376305 avg-key = 12.00 avg-zkey = 0.00 avg-val = 52.78 avg-zval = 17.33 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 4 UnZipSize{ index = 0.0885 GB value = 0.3893 GB dict = 3.90 MB all = 0.5369 GB } __ZipSize{ index = 0.0000 GB value = 0.1279 GB dict = 3.90 MB all = 0.1318 GB } UnZip/Zip{ index =461019.0625 value = 3.0453 dict = 1.00 all = 4.0749 } Zip/UnZip{ index = 0.0000 value = 0.3284 dict = 1.00 all = 0.2454 } ---------------------------- total value len = 115.748007 GB avg = 0.172 KB (by entry num) total key len = 14.440832 GB avg = 0.021 KB total ukey len = 8.071217 GB avg = 0.010 KB total ukey num = 0.782509450 Billion total entry num = 0.673960896 Billion write speed all = 37.912422498 MB/sec (with seq num) write speed all = 36.342303855 MB/sec (without seq num) 2024/07/08-15:39:17.217061 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628398: 7376305 keys, 131750269 bytes, temperature: kWarm 2024/07/08-15:39:17.217132 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424357217097, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628398, "file_size": 131750269, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 127850784, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 147526100, "raw_average_key_size": 20, "raw_value_size": 389344860, "raw_average_value_size": 52, "num_data_blocks": 1, "num_entries": 7376305, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1720424342, "oldest_key_time": 0, "file_creation_time": 1720424342, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628398, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:39:17.299158 139818989889088 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628399.sst) /dev/shm/ToplingTemp/Topling-21995-umtJ4l 2024/07/08-15:39:19.212736 139818989889088 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628399.sst): first pass time = 1.91's, 65.809'MB/sec 2024/07/08-15:39:20.657695 139818989889088 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628399.sst): old prealloc_size = 590558003, real_size = 33458768 2024/07/08-15:39:20.657745 139818989889088 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628399.sst): index type = UintIndex_AllOne , store type = DictZipBlobStore usrkeys = 1731155 prefix = 4 raw-key = 0.0138 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.00 raw-val = 0.0913 GB zip-val = 0.0330 GB avg-val = 52.74 avg-zval = 19.08 2024/07/08-15:39:21.881154 139818989889088 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00/12628399.sst): second pass time = 1.44's, 63.240'MB/sec, value only(72.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.22's, 28.010'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 9.141'MB/sec (index lex order gen) rebuild zvType time = 1.08's, 0.401'MB/sec write SST data time = 0.14's, 239.299'MB/sec dict compress time = 4049643381.98's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.911'MB, throughput = 22655.904'MB/sec zip my value time = 1.44's, unzip length = 0.091'GB zip my value throughput = 63.240'MB/sec zip pipeline throughput = 63.240'MB/sec entries = 1731155 avg-key = 20.00 avg-zkey = 0.00 avg-val = 52.74 avg-zval = 19.08 usrkeys = 1731155 avg-key = 12.00 avg-zkey = 0.00 avg-val = 52.83 avg-zval = 19.08 TagRS{ kinds = RS_Key0_TagN bytes = 270600 } TagArray{ size = 22905 bytes = 60144 } seq expand size = 160335 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0208 GB value = 0.0913 GB dict = 0.91 MB all = 0.1259 GB } __ZipSize{ index = 0.0000 GB value = 0.0330 GB dict = 0.91 MB all = 0.0343 GB } UnZip/Zip{ index =108197.1875 value = 2.7646 dict = 1.00 all = 3.6747 } Zip/UnZip{ index = 0.0000 value = 0.3617 dict = 1.00 all = 0.2721 } ---------------------------- total value len = 115.839469 GB avg = 0.171 KB (by entry num) total key len = 14.475455 GB avg = 0.021 KB total ukey len = 8.091991 GB avg = 0.010 KB total ukey num = 0.784240605 Billion total entry num = 0.675692051 Billion write speed all = 37.895934146 MB/sec (with seq num) write speed all = 36.323989466 MB/sec (without seq num) 2024/07/08-15:39:21.924704 139818989889088 [compaction_job.cc:1944] [default] [JOB 3530] Generated table #12628399: 1731155 keys, 34268617 bytes, temperature: kWarm 2024/07/08-15:39:21.924775 139818989889088 EVENT_LOG_v1 {"time_micros": 1720424361924739, "cf_name": "default", "job": 3530, "event": "table_file_creation", "file_number": 12628399, "file_size": 34268617, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 147194473013, "table_properties": {"data_size": 33025632, "index_size": 192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 34623100, "raw_average_key_size": 20, "raw_value_size": 91302589, "raw_average_value_size": 52, "num_data_blocks": 1, "num_entries": 1731155, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-UintIndex_AllOne-DictZipBlobStore", "creation_time": 1720424357, "oldest_key_time": 0, "file_creation_time": 1720424357, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12628399, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-15:39:21.946608 139818989889088 [compaction_job.cc:699] job-03530: subcompact[0], size: 3.400444 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03530/att-00 [12628383,12628384,12628385,12628386,12628387,12628388,12628389,12628390,12628391,12628392,12628393,12628394,12628395,12628396,12628397,12628398,12628399] 2024/07/08-15:39:22.331455 139818989889088 (Original Log Time 2024/07/08-15:35:52.549389) EVENT_LOG_v1 {"time_micros": 1720424152549026, "job": 3530, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L4": [12626359, 12626360, 12628316, 12626363, 12626534, 12626535, 12626536, 12626537], "files_L5": [12628379, 12628380, 12620036, 12619621, 12628318, 12625776, 12545820, 12610894, 12545672, 12575517, 12575518, 12391997, 12392105, 12392104, 12392225, 12392226, 12392332, 12392583, 12392707, 12392836, 12392935, 12393047, 12393162, 12393163, 12393429, 12393586, 12393735, 12393823, 12393830, 12394027, 12394032, 12394150, 12394241, 12394425, 12394530, 12394655, 12394738, 12394766, 12394905, 12394986, 12395002, 12395152, 12395254, 12395253, 12395359, 12395360, 12395457, 12395620, 12395777, 12395911, 12396032, 12396179, 12396319, 12396452, 12396551, 12396645, 12396644, 12396728, 12396726, 12396833, 12396933, 12396934, 12397052, 12397318, 12397397, 12397538, 12398283, 12398289, 12398404, 12398401, 12398499, 12398500, 12398603, 12398730, 12398831, 12398936, 12398934, 12399055, 12399056, 12399218, 12399355, 12399478, 12399568, 12399686, 12399685, 12399803, 12399940, 12400096, 12400118, 12400460, 12400775, 12401082, 12401382, 12401699, 12401943, 12402037, 12402140, 12402234, 12402325, 12402336, 12402425, 12402534, 12402650, 12402652, 12402765, 12402878, 12402879, 12403000, 12403130, 12403134, 12403217, 12403216, 12403299, 12403290, 12403394, 12403393, 12403495, 12403492, 12403595, 12403593, 12403688, 12403689, 12403798, 12403799, 12403910, 12403991, 12403993, 12404119, 12404230, 12404238, 12404319, 12404415, 12404416, 12404507, 12404506, 12404607, 12404602, 12404701, 12404702, 12404813, 12404814, 12404933, 12405044, 12405045, 12405161, 12405284, 12405256, 12405394, 12405480, 12405573, 12405671, 12405797, 12405792, 12405893, 12405963, 12406054, 12406168, 12406266, 12406361, 12406487, 12406579, 12406697, 12406787, 12406886, 12406986, 12406985, 12407086, 12407174, 12407294, 12407391, 12407520, 12407590, 12407684, 12407776, 12407873, 12407978, 12408085, 12408184, 12408289, 12408366, 12408465, 12408565, 12408674, 12408798, 12408929, 12409021, 12409143, 12409221, 12409320, 12409415, 12409486, 12409596, 12409595, 12409715, 12409823, 12409916, 12410006, 12410129, 12410204, 12410404, 12410496, 12410582, 12410581, 12410688, 12410689, 12410787, 12410882, 12411002, 12411095, 12411193, 12411274, 12411275, 12411383, 12411470, 12411575, 12411691, 12411796, 12411911, 12412012, 12412110, 12412252, 12412319, 12412318, 12412419, 12412543, 12412670, 12412794, 12412797, 12412935, 12413039, 12413136, 12413141, 12413247, 12413371, 12413468, 12413599, 12413698, 12413797, 12413944, 12414038, 12414154, 12414248, 12414352, 12414436, 12414429, 12414551, 12414687, 12414795, 12414882, 12414979, 12415089, 12415088, 12415197, 12415302, 12415420, 12415528, 12415650, 12415659, 12415749, 12415889, 12415999, 12416104, 12416220, 12416291, 12416459, 12416458, 12416462, 12416596, 12416735, 12416858, 12416951, 12417085, 12417086, 12417196, 12417328, 12417427, 12417560, 12417679, 12417770, 12417899, 12417986, 12418106, 12418207, 12418295, 12418408, 12418529, 12418607, 12418606, 12418726, 12418864, 12418963, 12419069, 12419066, 12419218, 12419306, 12419491, 12419557, 12419654, 12419777, 12419919, 12420041, 12420146, 12420317, 12420384, 12420582, 12420594, 12420898, 12420897, 12421018, 12421128, 12421233, 12421326, 12421451, 12421551, 12421663, 12421751, 12421972, 12422010, 12422119, 12422258, 12422353, 12422472, 12422563, 12422685, 12422689, 12422829, 12422952, 12423059, 12423210, 12423272, 12423414, 12423551, 12423625, 12423720, 12423723, 12423840, 12423981, 12424066, 12424167, 12424256, 12424364, 12424374, 12424487, 12424605, 12424734, 12424870, 12425069, 12425070, 12425289, 12425431, 12425541, 12425699, 12425785, 12425789, 12425893, 12425985, 12426082, 12426080, 12426187, 12426319, 12426426, 12426539, 12426637, 12426809, 12426877, 12426996, 12427114, 12427213, 12427335, 12427455, 12427569, 12427698, 12427783, 12427894, 12428032, 12428181, 12428275, 12428410, 12428500, 12428613, 12428617, 12428710, 12428842, 12428938, 12429029, 12429030, 12429140, 12429234, 12429369, 12429481, 12429618, 12429726, 12429850, 12429973, 12430111, 12430205, 12430324, 12430474, 12430633, 12430758, 12430870, 12431000, 12431131, 12431258, 12431420, 12431421, 12431543, 12431669, 12431821, 12431956, 12432088, 12432190, 12432265, 12432394, 12432399, 12432535, 12432657, 12432765, 12432870, 12432886, 12432989, 12433099, 12433097, 12433205, 12433329, 12433434, 12433526, 12433658, 12433759, 12433760, 12433861, 12433867, 12433957, 12433956, 12434045, 12434165, 12434176, 12434334, 12434437, 12434557, 12434677, 12434781, 12434886, 12434998, 12435102, 12435223, 12435328, 12435435, 12435545, 12435661, 12435731, 12435863, 12436023, 12436098, 12436214, 12436303, 12436406, 12436410, 12436494, 12436497, 12436602, 12436687, 12436798, 12436880, 12436974, 12436976, 12437088, 12437232, 12437367, 12437506, 12437586, 12437685, 12437768, 12437886, 12437977, 12438078, 12438168, 12438304, 12438431, 12438541, 12438665, 12438781, 12438894, 12439007, 12439126, 12439231, 12439319, 12439423, 12439535, 12439647, 12439646, 12439791, 12439908, 12439894, 12439990, 12440090, 12440216, 12440319, 12440426, 12440528, 12440628, 12440861, 12441008, 12441117, 12441291, 12441292, 12441293, 12441456, 12441551, 12441550, 12441676, 12441805, 12441961, 12442060, 12442206, 12442466, 12442456, 12442459, 12442668, 12442772, 12442889, 12442993, 12443087, 12443084, 12443188, 12443286, 12443312, 12443420, 12443428, 12443531, 12443644, 12443755, 12443869, 12443972, 12444074, 12444197, 12444341, 12444446, 12444595, 12444724, 12444824, 12444927, 12445031, 12445028, 12445131, 12445237, 12445323, 12445460, 12445580, 12445687, 12445815, 12445973, 12446059, 12446147, 12446246, 12446247, 12446400, 12446492, 12446495, 12446607, 12446690, 12446689, 12446798, 12446943, 12447090, 12447199, 12447345, 12447464, 12447465, 12447589, 12447686, 12447685, 12447768, 12447774, 12447905, 12448066, 12448186, 12448281, 12448283, 12448399, 12448504, 12448503, 12448613, 12448744, 12448838, 12448942, 12449080, 12449211, 12449344, 12449444, 12449443, 12449546, 12449666, 12449781, 12449783, 12449915, 12450020, 12450127, 12450242, 12450335, 12450445, 12450444, 12450577, 12450693, 12450692, 12450784, 12450805, 12450899, 12451002, 12451005, 12451134, 12451247, 12451388, 12451511, 12451573, 12451589, 12451678, 12451787, 12451890, 12451995, 12451994, 12452106, 12452222, 12452351, 12452444, 12452580, 12452681, 12452682, 12452780, 12452897, 12453001, 12453116, 12453243, 12453359, 12453500, 12453677, 12453765, 12453897, 12454029, 12454136, 12454266, 12454378, 12454468, 12454467, 12454554, 12454655, 12454765, 12454862, 12454869, 12454984, 12455080, 12455180, 12455178, 12455294, 12455293, 12455399, 12455550, 12455673, 12455757, 12455862, 12455953, 12456066, 12456175, 12456363, 12456462, 12456631, 12456765, 12456907, 12457041, 12457141, 12457255, 12457375, 12457498, 12457607, 12457740, 12457863, 12457976, 12458151, 12458241, 12458372, 12458533, 12458608, 12458709, 12458819, 12458931, 12459020, 12459018, 12459133, 12459300, 12459446, 12459536, 12459555, 12459652, 12459657, 12459786, 12459886, 12459891, 12459999, 12460091, 12460226, 12460390, 12460475, 12460480, 12460594, 12460596, 12460698, 12460788, 12460934, 12461066, 12461179, 12461300, 12461424, 12461533, 12461660, 12461767, 12461768, 12461869, 12461978, 12462079, 12462191, 12462283, 12462392, 12462474, 12462600, 12462602, 12462688, 12462806, 12462910, 12462911, 12463021, 12463114, 12463112, 12463229, 12463341, 12463340, 12463471, 12463542, 12463644, 12463649, 12463756, 12463850, 12463946, 12464075, 12464180, 12464310, 12464448, 12464555, 12464554, 12464688, 12464844, 12464986, 12465075, 12465080, 12465210, 12465211, 12465324, 12465325, 12465430, 12465565, 12465576, 12465681, 12465687, 12465808, 12465933, 12466014, 12466013, 12466126, 12466234, 12466337, 12466348, 12466478, 12466620, 12466724, 12466811, 12466917, 12467078, 12467198, 12467292, 12467427, 12467539, 12467671, 12467774, 12467923, 12468063, 12468167, 12468299, 12468410, 12468413, 12468575, 12468651, 12468749, 12468752, 12468843, 12469036, 12469130, 12469244, 12469351, 12469444, 12469541, 12469645, 12469642, 12469745, 12469751, 12469850, 12469942, 12470064, 12470205, 12470311, 12470438, 12470554, 12470631, 12470660, 12470727, 12470825, 12470951, 12471073, 12471173, 12471269, 12471380, 12471381, 12471483, 12471588, 12471587, 12471680, 12471682, 12471802, 12471803, 12471888, 12471999, 12471997, 12472150, 12472208, 12472207, 12472311, 12472446, 12472554, 12472553, 12472667, 12472665, 12472773, 12472772, 12472863, 12472859, 12472987, 12473066, 12473067, 12473187, 12473272, 12473270, 12473365, 12473486, 12473618, 12473717, 12473719, 12473867, 12473970, 12473978, 12474082, 12474188, 12474291, 12474299, 12474395, 12474407, 12474519, 12474688, 12474823, 12474899, 12474898, 12475027, 12475108, 12475217, 12475322, 12475459, 12475594, 12475719, 12475841, 12475953, 12476074, 12476077, 12476221, 12476354, 12476491, 12476490, 12476593, 12476706, 12476705, 12476875, 12476963, 12476961, 12477100, 12477161, 12477330, 12477431, 12477453, 12477582, 12477723, 12477806, 12477897, 12478029, 12478157, 12478274, 12478393, 12478495, 12478594, 12478596, 12478723, 12478895, 12478996, 12479118, 12479366, 12479370, 12479371, 12479497, 12479495, 12479642, 12479757, 12479891, 12479889, 12480017, 12480130, 12480283, 12480527, 12480526, 12480721, 12480865, 12480991, 12481128, 12481245, 12481367, 12481500, 12481685, 12481795, 12481929, 12482068, 12482197, 12482319, 12482427, 12482527, 12482638, 12482773, 12482872, 12482964, 12482965, 12483066, 12483161, 12483285, 12483433, 12483514, 12483617, 12483618, 12483729, 12483727, 12483849, 12483961, 12483962, 12484065, 12484067, 12484153, 12484158, 12484255, 12484257, 12484382, 12484514, 12484615, 12484616, 12484724, 12484800, 12484805, 12484910, 12485006, 12485117, 12485240, 12485367, 12485477, 12485567, 12485709, 12485827, 12485936, 12486038, 12486037, 12486138, 12486139, 12486257, 12486361, 12486486, 12486484, 12486567, 12486673, 12486677, 12486762, 12486906, 12486998, 12487130, 12487242, 12487379, 12487498, 12487633, 12487779, 12487916, 12488007, 12488141, 12488258, 12488341, 12488342, 12488437, 12488436, 12488545, 12488540, 12488636, 12488638, 12488734, 12488837, 12488945, 12488943, 12489038, 12489133, 12489131, 12489237, 12489225, 12489322, 12489341, 12489439, 12489590, 12489593, 12489686, 12489764, 12489763, 12489860, 12489957, 12489955, 12490107, 12490230, 12490359, 12490493, 12490494, 12490585, 12490593, 12490687, 12490685, 12490788, 12490888, 12490891, 12491002, 12491140, 12491231, 12491330, 12491439, 12491437, 12491527, 12491530, 12491647, 12491738, 12491739, 12491841, 12491846, 12491972, 12492073, 12492186, 12492309, 12492310, 12492407, 12492408, 12492509, 12492620, 12492717, 12492718, 12492805, 12492919, 12493002, 12493147, 12493258, 12493256, 12493353, 12493491, 12493595, 12493696, 12493835, 12493904, 12494037, 12494167, 12494261, 12494364, 12494465, 12494509, 12494598, 12494711, 12494824, 12494820, 12494942, 12495088, 12495202, 12495302, 12495402, 12495520, 12495605, 12495602, 12495753, 12495879, 12495980, 12496072, 12496200, 12496194, 12496329, 12496328, 12496430, 12496562, 12496668, 12496667, 12496794, 12496928, 12497138, 12497161, 12497259, 12497395, 12497515, 12497624, 12497711, 12497824, 12497935, 12498050, 12498052, 12498149, 12498253, 12498373, 12498482, 12498618, 12498747, 12498748, 12498864, 12498979, 12498978, 12499104, 12499268, 12499367, 12499460, 12499564, 12499565, 12499666, 12499769, 12499891, 12500006, 12500110, 12500209, 12500214, 12500347, 12500466, 12500465, 12500600, 12500677, 12500676, 12500780, 12500917, 12501014, 12501125, 12501268, 12501400, 12501539, 12501629, 12501754, 12501901, 12501995, 12501993, 12502132, 12502249, 12502374, 12502458, 12502460, 12502552, 12502661, 12502766, 12502767, 12502870, 12502881, 12502977, 12503122, 12503178, 12503180, 12503275, 12503381, 12503523, 12503638, 12503639, 12503740, 12503741, 12503846, 12503949, 12504055, 12504160, 12504260, 12504373, 12504370, 12504492, 12504662, 12504793, 12505009, 12505008, 12505014, 12505191, 12505190, 12505289, 12505288, 12505386, 12505486, 12505590, 12505692, 12505784, 12505902, 12505979, 12506071, 12506069, 12506168, 12506169, 12506269, 12506274, 12506370, 12506368, 12506490, 12506586, 12506707, 12506811, 12506803, 12506937, 12507044, 12507143, 12507247, 12507338, 12507339, 12507441, 12507554, 12507633, 12507739, 12507858, 12508009, 12508094, 12508173, 12508185, 12508286, 12508391, 12508495, 12508598, 12508718, 12508835, 12508959, 12509087, 12509185, 12509309, 12509393, 12509501, 12509623, 12509693, 12509788, 12509888, 12509992, 12509991, 12510121, 12510233, 12510232, 12510325, 12510433, 12510437, 12510554, 12510638, 12510639, 12510736, 12510733, 12510839, 12510979, 12511085, 12511083, 12511192, 12511284, 12511392, 12511380, 12511504, 12511614, 12511613, 12511737, 12511736, 12511864, 12511986, 12512103, 12512110, 12512190, 12512191, 12512322, 12512421, 12512489, 12512490, 12512616, 12512594, 12512745, 12512859, 12512990, 12513101, 12513240, 12513350, 12513349, 12513448, 12513540, 12513650, 12513651, 12513761, 12513848, 12513847, 12513947, 12514072, 12514198, 12514362, 12514471, 12514474, 12514572, 12514717, 12514795, 12514911, 12514998, 12514997, 12515099, 12515100, 12515239, 12515360, 12515472, 12515575, 12515691, 12515842, 12515974, 12516085, 12516192, 12516191, 12516290, 12516298, 12516435, 12516535, 12516531, 12516684, 12516803, 12516929, 12517069, 12517187, 12517202, 12517339, 12517451, 12517452, 12517545, 12517544, 12517669, 12517668, 12517766, 12517767, 12517872, 12517979, 12518075, 12518074, 12518195, 12518193, 12518291, 12518290, 12518393, 12518389, 12518507, 12518617, 12518619, 12518778, 12518864, 12518865, 12518973, 12518972, 12519071, 12519177, 12519315, 12519470, 12519552, 12519655, 12519775, 12519885, 12519990, 12520090, 12520186, 12520289, 12520382, 12520497, 12520500, 12520610, 12520711, 12520816, 12520948, 12520954, 12521052, 12521184, 12521320, 12521459, 12521589, 12521720, 12521816, 12521947, 12521951, 12522093, 12522229, 12522335, 12522467, 12522556, 12522692, 12522788, 12522786, 12522891, 12522991, 12522995, 12523129, 12523200, 12523329, 12523408, 12523407, 12523542, 12523632, 12523717, 12523826, 12523917, 12524030, 12524117, 12524222, 12524375, 12524477, 12524591, 12524720, 12524805, 12524903, 12525004, 12525097, 12525239, 12525344, 12525446, 12525535, 12525636, 12525638, 12525759, 12525922, 12525998, 12526121, 12526120, 12526228, 12526212, 12526316, 12526318, 12526420, 12526419, 12526521, 12526660, 12526748, 12526852, 12526955, 12526957, 12527091, 12527155, 12527253, 12527258, 12527367, 12527459, 12527569, 12527644, 12527770, 12527858, 12527857, 12527957, 12528067, 12528175, 12528253, 12528258, 12528353, 12528352, 12528474, 12528562, 12528665, 12528764, 12528763, 12528869, 12528968, 12528967, 12529062, 12529186, 12529301, 12529398, 12529397, 12529502, 12529601, 12529714, 12529812, 12529961, 12529952, 12530073, 12530185, 12530300, 12530426, 12530427, 12530526, 12530533, 12530623, 12530729, 12530828, 12530931, 12530929, 12531032, 12531026, 12531138, 12531244, 12531389, 12531390, 12531481, 12531571, 12531667, 12531669, 12531772, 12531895, 12531982, 12532116, 12532213, 12532397, 12532499, 12532611, 12532609, 12532708, 12532707, 12532817, 12532905, 12533008, 12533114, 12533232, 12533319, 12533422, 12533423, 12533524, 12533626, 12533751, 12533724, 12533839, 12533925, 12533924, 12534035, 12534036, 12534136, 12534135, 12534242, 12534342, 12534489, 12534615, 12534614, 12534723, 12534842, 12534958, 12535054, 12535055, 12535153, 12535263, 12535264, 12535422, 12535506, 12535508, 12535608, 12535609, 12535722, 12535830, 12535987, 12536096, 12536175, 12536278, 12536277, 12536421, 12536527, 12536650, 12536763, 12536762, 12536896, 12537027, 12537140, 12537141, 12537227, 12537348, 12537468, 12537550, 12537665, 12537812, 12537904, 12537902, 12538014, 12538012, 12538130, 12538230, 12538370, 12538477, 12538587, 12538684, 12538683, 12538805, 12538928, 12539097, 12539062, 12539170, 12539323, 12539452, 12539563, 12539697, 12539698, 12539843, 12539964, 12540078, 12540206, 12540317, 12540458, 12540541, 12540638, 12540641, 12540740, 12540868, 12540988, 12540989, 12541110, 12541240, 12541243, 12541367, 12541485, 12541621, 12541753, 12541748, 12541840, 12541946, 12542070, 12542043, 12542151, 12542310, 12542390, 12542391, 12542522, 12542605, 12542726, 12542822, 12542923, 12542921, 12543052, 12543051, 12543136, 12543235, 12543240, 12543390, 12543491, 12543610, 12543709, 12543707, 12543832, 12543967, 12544086, 12544214, 12544338, 12544482, 12544483, 12544605, 12544601, 12544718, 12544833, 12544919, 12544917, 12545029, 12545164, 12545297, 12545396, 12545394, 12545514, 12545609, 12545747, 12545894, 12546053, 12546199, 12546363, 12546478, 12546566, 12546702, 12546841, 12546838, 12546956, 12547078, 12547212, 12547208, 12547343, 12547479, 12547606, 12547607, 12547694, 12547809, 12547997, 12548114, 12548210, 12548369, 12548474, 12548610, 12548718, 12548824, 12548931, 12549061, 12549186, 12549293, 12549402, 12549539, 12549540, 12549659, 12549666, 12549768, 12549767, 12549875, 12549977, 12550156, 12550276, 12550467, 12550552, 12550550, 12550669, 12550765, 12550868, 12550999, 12550974, 12551114, 12551234, 12551368, 12551486, 12551487, 12551609, 12551723, 12551843, 12551838, 12551952, 12552072, 12552168, 12552269, 12552271, 12552372, 12552486, 12552487, 12552591, 12552740, 12552857, 12552974, 12553095, 12553096, 12553181, 12553344, 12553454, 12553588, 12553739, 12553854, 12553952, 12553959, 12554060, 12554199, 12554321, 12554420, 12554523, 12554616, 12554623, 12554738, 12554735, 12554865, 12554981, 12554985, 12555119, 12555227, 12555356, 12555359, 12555464, 12555566, 12555567, 12555677, 12555779, 12555780, 12555909, 12555996, 12556077, 12556189, 12556190, 12556276, 12556277, 12556386, 12556384, 12556479, 12556629, 12556761, 12556832, 12556955, 12557045, 12557044, 12557139, 12557147, 12557260, 12557375, 12557489, 12557604, 12557603, 12557713, 12557817, 12557912, 12558018, 12558034, 12558177, 12558269, 12558275, 12558385, 12558499, 12558668, 12558779, 12558901, 12559011, 12559010, 12559115, 12559258, 12559390, 12559531, 12559642, 12559773, 12559891, 12560017, 12560151, 12560261, 12560398, 12560525, 12560626, 12560627, 12560758, 12560874, 12561049, 12561050, 12561191, 12561195, 12561316, 12561433, 12561530, 12561534, 12561645, 12561658, 12561751, 12561762, 12561884, 12562003, 12562004, 12562115, 12562113, 12562229, 12562347, 12562465, 12562464, 12562642, 12562786, 12562868, 12562869, 12562971, 12562969, 12563071, 12563092, 12563179, 12563202, 12563414, 12563413, 12563557, 12563664, 12563665, 12563756, 12563752, 12563850, 12563851, 12563971, 12563972, 12564053, 12564176, 12564309, 12564434, 12564538, 12564541, 12564660, 12564771, 12564780, 12564897, 12565025, 12565164, 12565305, 12565415, 12565516, 12565627, 12565626, 12565722, 12565831, 12565849, 12565942, 12566132, 12566249, 12566373, 12566463, 12566588, 12566735, 12566849, 12566846, 12566951, 12567109, 12567203, 12567199, 12567340, 12567461, 12567460, 12567593, 12567732, 12567861, 12568000, 12568086, 12568085, 12568191, 12568201, 12568331, 12568529, 12568536, 12568707, 12568709, 12568818, 12568992, 12569108, 12569298, 12569299, 12569368, 12569508, 12569667, 12569785, 12569899, 12569911, 12570022, 12570023, 12570133, 12570223, 12570381, 12570487, 12570583, 12570683, 12570684, 12570789, 12570925, 12571082, 12571207, 12571333, 12571469, 12571608, 12571744, 12571879, 12571986, 12571987, 12572157, 12572314, 12572410, 12572417, 12572526, 12572522, 12572686, 12572830, 12572944, 12573025, 12573149, 12573315, 12573410, 12573483, 12573565, 12573698, 12573821, 12573941, 12574056, 12574058, 12574185, 12574295, 12574301, 12574421, 12574422, 12574511, 12574612, 12574697, 12574699, 12574812, 12574811, 12574897, 12574896, 12575026, 12575097, 12575200, 12575204, 12575297, 12575434, 12575557, 12575559, 12575689, 12575796, 12575795, 12575897, 12575988, 12576124, 12576259, 12576445, 12576549, 12576646, 12576650, 12576785, 12576910, 12576928, 12577048, 12577167, 12577173, 12577270, 12577274, 12577380, 12577377, 12577485, 12577595, 12577597, 12577782, 12577894, 12577901, 12578031, 12578163, 12578314, 12578442, 12578440, 12578563, 12578694, 12578800, 12578921, 12579006, 12579164, 12579306, 12579423, 12579530, 12579667, 12579788, 12579950, 12580085, 12580258, 12580260, 12580430, 12580423, 12580551, 12580687, 12580778, 12580900, 12580992, 12581123, 12581252, 12581381, 12581492, 12581642, 12581790, 12581887, 12581987, 12581982, 12582098, 12582216, 12582348, 12582458, 12582459, 12582559, 12582560, 12582648, 12582647, 12582765, 12582882, 12583025, 12583133, 12583272, 12583268, 12583380, 12583353, 12583435, 12583596, 12583729, 12583839, 12583974, 12584130, 12584131, 12584301, 12584378, 12584484, 12584621, 12584740, 12584843, 12584936, 12584938, 12585048, 12585162, 12585265, 12585277, 12585365, 12585467, 12585465, 12585583, 12585689, 12585677, 12585837, 12585954, 12586067, 12586192, 12586157, 12586332, 12586428, 12586437, 12586560, 12586556, 12586700, 12586810, 12586809, 12586918, 12587020, 12587151, 12587305, 12587526, 12587527, 12587663, 12587756, 12587880, 12587877, 12588015, 12588124, 12588121, 12588248, 12588371, 12588493, 12588608, 12588752, 12588926, 12589031, 12589147, 12589243, 12589378, 12589533, 12589663, 12589794, 12589912, 12589911, 12590129, 12590130, 12590139, 12590138, 12590302, 12590383, 12590485, 12590500, 12590583, 12590705, 12590794, 12590902, 12590904, 12591027, 12591129, 12591233, 12591236, 12591332, 12591452, 12591540, 12591660, 12591770, 12591856, 12591973, 12592116, 12592114, 12592249, 12592354, 12592495, 12592600, 12592749, 12592843, 12592844, 12592982, 12593093, 12593211, 12593210, 12593305, 12593425, 12593643, 12593648, 12593794, 12593959, 12593968, 12594078, 12594188, 12594293, 12594291, 12594390, 12594394, 12594537, 12594644, 12594643, 12594749, 12594751, 12594858, 12594955, 12594975, 12595161, 12595201, 12595361, 12595494, 12595587, 12595713, 12595894, 12595990, 12596128, 12596341, 12596340, 12596438, 12596534, 12596535, 12596644, 12596652, 12596747, 12596763, 12596850, 12597002, 12597097, 12597235, 12597334, 12597419, 12597518, 12597654, 12597785, 12597921, 12598027, 12598016, 12598154, 12598302, 12598443, 12598575, 12598674, 12598779, 12598773, 12598890, 12599019, 12599018, 12599131, 12599224, 12599226, 12599352, 12599475, 12599477, 12599610, 12599768, 12599878, 12600026, 12600138, 12600282, 12600374, 12600478, 12600592, 12600585, 12600691, 12600688, 12600797, 12600795, 12600906, 12601015, 12601110, 12601109, 12601212, 12601348, 12601482, 12601588, 12601586, 12601672, 12601675, 12601798, 12601892, 12601999, 12602006, 12602105, 12602104, 12602204, 12602317, 12602422, 12602576, 12602676, 12602819, 12602925, 12602923, 12603062, 12603174, 12603310, 12603445, 12603575, 12603719, 12603860, 12603964, 12603962, 12604109, 12604112, 12604249, 12604247, 12604357, 12604491, 12604490, 12604602, 12604741, 12604865, 12604968, 12605067, 12605160, 12605163, 12605272, 12605408, 12605528, 12605531, 12605625, 12605722, 12605862, 12605997, 12605993, 12606102, 12606267, 12606269, 12606348, 12606503, 12606603, 12606737, 12606859, 12606855, 12606998, 12607110, 12607113, 12607220, 12607219, 12607319, 12607421, 12607542, 12607669, 12607768, 12607767, 12607880, 12607881, 12607988, 12607987, 12608091, 12608199, 12608296, 12608405, 12608533, 12608660, 12608658, 12608758, 12608887, 12608888, 12608993, 12609126, 12609137, 12609258, 12609369, 12609479, 12609476, 12609622, 12609740, 12609739, 12609865, 12609940, 12610096, 12610209, 12610338, 12610467, 12610588, 12610577, 12610680, 12610815, 12610937, 12611078, 12611193, 12611291, 12611403, 12611406, 12611514, 12611630, 12611714, 12611841, 12611843, 12611987, 12612161, 12612244, 12612367, 12612506, 12612646, 12612793, 12612918, 12613054, 12613167, 12613286, 12613439, 12613534, 12613695, 12613816, 12613947, 12614081, 12614183, 12614185, 12614305, 12614401, 12614613, 12614612, 12614801, 12614904, 12614991, 12615093, 12615095, 12615246, 12615362, 12615363, 12615491, 12615597, 12615595, 12615699, 12615820, 12615976, 12615956, 12616063, 12616200, 12616213, 12616293, 12616413, 12616559, 12616659, 12616801, 12616914, 12617026, 12617193, 12617282, 12617283, 12617424, 12617532, 12617666, 12617805, 12617927, 12618014, 12618199, 12618327, 12618486, 12618599, 12618870, 12618892, 12618891, 12619087, 12619195, 12619209, 12619302, 12619426, 12619407, 12619540, 12619661, 12619778, 12619875, 12619982, 12620079, 12620185, 12620184, 12620332, 12620452, 12620448, 12620563, 12620562, 12620670, 12620799, 12620813, 12620934, 12621113, 12621259, 12621371, 12621537, 12621659, 12621775, 12621899, 12621996, 12622109, 12622111, 12622213, 12622212, 12622326, 12622477, 12622646, 12622651, 12622737, 12622887, 12623004, 12623119, 12623238, 12623365, 12623361, 12623484, 12623584, 12623717, 12623716, 12623794, 12623898, 12623906, 12624048, 12624165, 12624164, 12624297, 12624452, 12624573, 12624692, 12624799, 12624801, 12624903, 12625047, 12625155, 12625278, 12625662, 12625960, 12625961, 12626051, 12626140, 12626206], "score": -1, "input_data_size": 4442425455, "oldest_snapshot_seqno": 147237044821} 2024/07/08-15:39:22.331499 139818989889088 (Original Log Time 2024/07/08-15:39:22.327592) [compaction_job.cc:2018] [default] [JOB 3530] Compacted 8@4 + 2508@5 files to L5 => 3400444490 bytes 2024/07/08-15:39:22.331500 139818989889088 (Original Log Time 2024/07/08-15:39:22.331100) [compaction_job.cc:1220] [default] compacted to: files[0 0 0 0 0 17 0] max score 0.00, MB/sec: 21.2 rd, 16.2 wr, level 5, files in(8, 2508) out(17 +0 blob) MB in(989.8, 3246.8 +0.0 blob) out(3242.9 +0.0 blob), read-write-amplify(7.6) write-amplify(3.3) OK, records in: 58660253, records dropped: 2729662 output_compression: Snappy 2024/07/08-15:39:22.331504 139818989889088 (Original Log Time 2024/07/08-15:39:22.331333) EVENT_LOG_v1 {"time_micros": 1720424362331129, "cf": "default", "job": 3530, "event": "compaction_finished", "compaction_time_micros": 209397224, "compaction_time_cpu_micros": 47139070, "output_level": 5, "num_output_files": 17, "total_output_size": 3400444490, "num_input_records": 58660253, "num_output_records": 55930591, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 2032418128, "file_range_sync_nanos": 0, "file_fsync_nanos": 13683, "file_prepare_write_nanos": 41725, "lsm_state": [0, 0, 0, 0, 0, 17, 0]} 2024/07/08-15:39:22.489098 139818989889088 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"a007a635-5c0a-4b4a-be96-b9c695cb016b","dbStarts":1720420896,"starts":1720424152,"executesMs":210030,"compactionJobId":3530,"attempt":0,"compactionInputRawBytes":7945042028,"compactionInputZipBytes":4442425455,"compactionOutputRawBytes":0,"compactionOutputZipBytes":3400444490,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-6fed1a8d3498249bd0b755e925d3ab55-d3491ccac85852f6-00"}}, response = 2024/07/08-15:39:22.489225 139818989889088 INFO dcompact_worker.cpp:1456: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-03530/att-00: olev 5, work 210.031 s, result 7.002 ms, install 3.946 ms, input{raw 7.399 GiB zip 4.137 GiB}