2024/11/24-00:05:04.400199 139957248734784 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00: fork to child time = 0.007576 sec 2024/11/24-00:05:04.400897 139957248734784 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00 2024/11/24-00:05:04.409698 139957248734784 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00 2024/11/24-00:05:04.410593 139957248734784 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-00:05:04.410926 139957248734784 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-00:05:04.463795 139957248734784 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38296/att-00/MANIFEST-206347 2024/11/24-00:05:04.480415 139957248734784 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38296/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 285286, last_sequence is 4339636731, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 285182 2024/11/24-00:05:04.480421 139957248734784 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-00:05:04.480512 139957248734784 [version_set.cc:5721] Creating manifest 285286 2024/11/24-00:05:04.513877 139957248734784 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-00:05:04.533200 139957248734784 [compaction_job.cc:2437] [default] [JOB 38296] Compacting 1@2 + 13@3 files to L3, score 0.55, subcompactions 1 : 1 2024/11/24-00:05:04.533215 139957248734784 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 28699 Base level 2, inputs: [285250(27MB)], [285261(35MB) 285262(969KB) 285208(35MB) 285209(35MB) 285210(35MB) 285211(35MB) 285212(35MB) 285213(8689KB) 285214(35MB) 285215(35MB) 285216(970KB) 285192(35MB) 285274(35MB)] 2024/11/24-00:05:04.540539 139957248734784 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:05:04.540570 139957248734784 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:05:04.540590 139957248734784 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:05:04.540595 139957248734784 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:05:04.540606 139957248734784 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:05:05.169442 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285287.sst) /dev/shm/ToplingTemp/Topling-222928-Ih1up0 2024/11/24-00:05:05.723800 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285287.sst): first pass time = 0.55's, 242.134'MB/sec 2024/11/24-00:05:07.237166 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285287.sst): old prealloc_size = 147639500, real_size = 33564240 2024/11/24-00:05:07.237722 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285287.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925197 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.07 avg-zval = 34.12 2024/11/24-00:05:07.238511 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285287.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:05:08.101356 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285287.sst): second pass time = 1.51's, 71.631'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.443'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3368.782'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.315'MB/sec write SST data time = 0.13's, 288.369'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 23128.228'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.631'MB/sec zip pipeline throughput = 76.527'MB/sec entries = 925197 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.07 avg-zval = 34.12 usrkeys = 925197 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.07 avg-zval = 34.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925197 bytes = 3122560 } seq expand size = 6476379 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5109 value = 3.4306 dict = 1.00 all = 3.5751 } Zip/UnZip{ index = 0.0951 value = 0.2915 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 8.963408 GB avg = 0.168 KB (by entry num) total key len = 1.466727 GB avg = 0.028 KB total ukey len = 1.016382 GB avg = 0.015 KB total ukey num = 0.066475632 Billion total entry num = 0.053283361 Billion write speed all = 0.191042507 MB/sec (with seq num) write speed all = 0.183234833 MB/sec (without seq num) 2024/11/24-00:05:08.147808 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285287: 925197 keys, 37542272 bytes, temperature: kWarm 2024/11/24-00:05:08.147902 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377908147845, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285287, "file_size": 37542272, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178503, "largest_seqno": 4339359231, "table_properties": {"data_size": 31572288, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25905516, "raw_average_key_size": 28, "raw_value_size": 108312318, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925197, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377905, "oldest_key_time": 0, "file_creation_time": 1732377905, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285287, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:08.183314 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285288.sst) /dev/shm/ToplingTemp/Topling-222928-ArurDY 2024/11/24-00:05:09.097666 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285288.sst): first pass time = 0.91's, 146.796'MB/sec 2024/11/24-00:05:10.702349 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285288.sst): old prealloc_size = 147639500, real_size = 32815616 2024/11/24-00:05:10.702916 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285288.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930675 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.22 avg-zval = 33.09 2024/11/24-00:05:10.703703 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285288.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:05:11.566420 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285288.sst): second pass time = 1.60's, 67.437'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.599'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3301.500'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 274.193'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 26291.729'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.437'MB/sec zip pipeline throughput = 111.956'MB/sec entries = 930675 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.22 avg-zval = 33.09 usrkeys = 930675 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.22 avg-zval = 33.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930675 bytes = 3141040 } seq expand size = 6514725 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4437 value = 3.5116 dict = 1.00 all = 3.6465 } Zip/UnZip{ index = 0.0958 value = 0.2848 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 9.078082 GB avg = 0.167 KB (by entry num) total key len = 1.492785 GB avg = 0.028 KB total ukey len = 1.034995 GB avg = 0.015 KB total ukey num = 0.067406307 Billion total entry num = 0.054214036 Billion write speed all = 0.193607932 MB/sec (with seq num) write speed all = 0.185664389 MB/sec (without seq num) 2024/11/24-00:05:11.617122 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285288: 930675 keys, 36807754 bytes, temperature: kWarm 2024/11/24-00:05:11.617200 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377911617160, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285288, "file_size": 36807754, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178518, "largest_seqno": 4339359306, "table_properties": {"data_size": 30800496, "index_size": 1782272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058900, "raw_average_key_size": 28, "raw_value_size": 108158858, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930675, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377908, "oldest_key_time": 0, "file_creation_time": 1732377908, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285288, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:11.841138 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285289.sst) /dev/shm/ToplingTemp/Topling-222928-TUCyi0 2024/11/24-00:05:12.770278 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285289.sst): first pass time = 0.93's, 144.463'MB/sec 2024/11/24-00:05:14.270860 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285289.sst): old prealloc_size = 147639500, real_size = 33230784 2024/11/24-00:05:14.271433 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285289.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925013 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.10 avg-zval = 33.77 2024/11/24-00:05:14.272205 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285289.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:05:15.146311 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285289.sst): second pass time = 1.50's, 72.234'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.503'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3215.753'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.13's, 286.593'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33242.175'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.234'MB/sec zip pipeline throughput = 72.234'MB/sec entries = 925013 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 33.77 usrkeys = 925013 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 33.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925013 bytes = 3121936 } seq expand size = 6475091 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5127 value = 3.4673 dict = 1.00 all = 3.6072 } Zip/UnZip{ index = 0.0951 value = 0.2884 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 9.192875 GB avg = 0.167 KB (by entry num) total key len = 1.518686 GB avg = 0.028 KB total ukey len = 1.053496 GB avg = 0.015 KB total ukey num = 0.068331320 Billion total entry num = 0.055139049 Billion write speed all = 0.196171893 MB/sec (with seq num) write speed all = 0.188093345 MB/sec (without seq num) 2024/11/24-00:05:15.195968 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285289: 925013 keys, 37208414 bytes, temperature: kWarm 2024/11/24-00:05:15.196054 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377915196017, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285289, "file_size": 37208414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178548, "largest_seqno": 4339359499, "table_properties": {"data_size": 31239536, "index_size": 1759808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900364, "raw_average_key_size": 28, "raw_value_size": 108317434, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925013, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377911, "oldest_key_time": 0, "file_creation_time": 1732377911, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285289, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:15.233128 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285290.sst) /dev/shm/ToplingTemp/Topling-222928-DAceNX 2024/11/24-00:05:16.204691 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285290.sst): first pass time = 0.97's, 138.152'MB/sec 2024/11/24-00:05:17.701081 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285290.sst): old prealloc_size = 147639500, real_size = 33190752 2024/11/24-00:05:17.701712 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285290.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927178 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.76 avg-zval = 33.65 2024/11/24-00:05:17.702532 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285290.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:05:18.588569 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285290.sst): second pass time = 1.49's, 72.415'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.889'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2910.493'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 276.868'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34529.504'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.415'MB/sec zip pipeline throughput = 72.415'MB/sec entries = 927178 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 33.65 usrkeys = 927178 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 33.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927178 bytes = 3129248 } seq expand size = 6490246 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5136 value = 3.4703 dict = 1.00 all = 3.6104 } Zip/UnZip{ index = 0.0951 value = 0.2882 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 9.307622 GB avg = 0.166 KB (by entry num) total key len = 1.544647 GB avg = 0.028 KB total ukey len = 1.072039 GB avg = 0.015 KB total ukey num = 0.069258498 Billion total entry num = 0.056066227 Billion write speed all = 0.198736295 MB/sec (with seq num) write speed all = 0.190522422 MB/sec (without seq num) 2024/11/24-00:05:18.637466 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285290: 927178 keys, 37174840 bytes, temperature: kWarm 2024/11/24-00:05:18.637540 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377918637504, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285290, "file_size": 37174840, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178559, "largest_seqno": 4339359148, "table_properties": {"data_size": 31195024, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960984, "raw_average_key_size": 28, "raw_value_size": 108256744, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927178, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377915, "oldest_key_time": 0, "file_creation_time": 1732377915, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285290, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:18.673779 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285291.sst) /dev/shm/ToplingTemp/Topling-222928-iGyCPX 2024/11/24-00:05:19.673687 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285291.sst): first pass time = 1.00's, 134.236'MB/sec 2024/11/24-00:05:21.153483 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285291.sst): old prealloc_size = 147639500, real_size = 33204640 2024/11/24-00:05:21.154035 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285291.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928693 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.52 avg-zval = 33.59 2024/11/24-00:05:21.154785 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285291.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:05:22.012372 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285291.sst): second pass time = 1.48's, 73.190'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.304'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3349.111'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 285.282'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35253.976'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.190'MB/sec zip pipeline throughput = 73.190'MB/sec entries = 928693 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.52 avg-zval = 33.59 usrkeys = 928693 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.52 avg-zval = 33.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928693 bytes = 3134352 } seq expand size = 6500851 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4447 value = 3.4691 dict = 1.00 all = 3.6088 } Zip/UnZip{ index = 0.0957 value = 0.2883 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 9.422337 GB avg = 0.165 KB (by entry num) total key len = 1.570650 GB avg = 0.028 KB total ukey len = 1.090613 GB avg = 0.016 KB total ukey num = 0.070187191 Billion total entry num = 0.056994920 Billion write speed all = 0.201300636 MB/sec (with seq num) write speed all = 0.192951230 MB/sec (without seq num) 2024/11/24-00:05:22.063080 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285291: 928693 keys, 37192104 bytes, temperature: kWarm 2024/11/24-00:05:22.063156 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377922063118, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285291, "file_size": 37192104, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178575, "largest_seqno": 4339359162, "table_properties": {"data_size": 31194000, "index_size": 1778304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003404, "raw_average_key_size": 28, "raw_value_size": 108214334, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928693, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377918, "oldest_key_time": 0, "file_creation_time": 1732377918, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285291, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:22.097866 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285292.sst) /dev/shm/ToplingTemp/Topling-222928-JAINmX 2024/11/24-00:05:23.267446 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285292.sst): first pass time = 1.17's, 114.761'MB/sec 2024/11/24-00:05:24.619477 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285292.sst): old prealloc_size = 147639500, real_size = 33406208 2024/11/24-00:05:24.620193 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285292.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925108 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.08 avg-zval = 33.96 2024/11/24-00:05:24.620987 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285292.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:05:25.503007 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285292.sst): second pass time = 1.35's, 80.167'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.313'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2625.597'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.311'MB/sec write SST data time = 0.14's, 267.759'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35136.291'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.167'MB/sec zip pipeline throughput = 80.167'MB/sec entries = 925108 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 33.96 usrkeys = 925108 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 33.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925108 bytes = 3122256 } seq expand size = 6475756 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5122 value = 3.4479 dict = 1.00 all = 3.5902 } Zip/UnZip{ index = 0.0951 value = 0.2900 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 9.537127 GB avg = 0.165 KB (by entry num) total key len = 1.596553 GB avg = 0.028 KB total ukey len = 1.109115 GB avg = 0.016 KB total ukey num = 0.071112299 Billion total entry num = 0.057920028 Billion write speed all = 0.203863949 MB/sec (with seq num) write speed all = 0.195379563 MB/sec (without seq num) 2024/11/24-00:05:25.560923 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285292: 925108 keys, 37384024 bytes, temperature: kWarm 2024/11/24-00:05:25.561010 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377925560959, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285292, "file_size": 37384024, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4339359443, "table_properties": {"data_size": 31414704, "index_size": 1760064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903024, "raw_average_key_size": 28, "raw_value_size": 108314836, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925108, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377922, "oldest_key_time": 0, "file_creation_time": 1732377922, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285292, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:25.599230 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285293.sst) /dev/shm/ToplingTemp/Topling-222928-f8nWVZ 2024/11/24-00:05:26.198594 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285293.sst): first pass time = 0.60's, 75.953'MB/sec 2024/11/24-00:05:26.653590 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285293.sst): old prealloc_size = 147639500, real_size = 11704304 2024/11/24-00:05:26.653832 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285293.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 317611 prefix = 4 raw-key = 0.0051 GB zip-key = 0.0006 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0366 GB zip-val = 0.0110 GB avg-val = 115.32 avg-zval = 34.68 2024/11/24-00:05:26.654150 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285293.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0008 GB, waited 0.000 sec, Key+Value = 0.046 GB 2024/11/24-00:05:26.947170 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285293.sst): second pass time = 0.45's, 80.639'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.29's, 44.268'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2807.245'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.322'MB/sec write SST data time = 0.05's, 278.447'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.375'MB, throughput = 12354.811'MB/sec zip my value time = 0.45's, unzip length = 0.037'GB zip my value throughput = 80.639'MB/sec zip pipeline throughput = 80.639'MB/sec entries = 317611 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.32 avg-zval = 34.68 usrkeys = 317611 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.32 avg-zval = 34.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 317611 bytes = 992544 } seq expand size = 2223277 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0064 GB value = 0.0366 GB dict = 0.38 MB all = 0.0455 GB } __ZipSize{ index = 0.0006 GB value = 0.0110 GB dict = 0.38 MB all = 0.0130 GB } UnZip/Zip{ index = 10.4291 value = 3.3251 dict = 1.00 all = 3.5029 } Zip/UnZip{ index = 0.0959 value = 0.3007 dict = 1.00 all = 0.2855 } ---------------------------- total value len = 9.575978 GB avg = 0.164 KB (by entry num) total key len = 1.605446 GB avg = 0.028 KB total ukey len = 1.115468 GB avg = 0.016 KB total ukey num = 0.071429910 Billion total entry num = 0.058237639 Billion write speed all = 0.204732757 MB/sec (with seq num) write speed all = 0.196202071 MB/sec (without seq num) 2024/11/24-00:05:26.970467 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285293: 317611 keys, 12995037 bytes, temperature: kWarm 2024/11/24-00:05:26.970537 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377926970503, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285293, "file_size": 12995037, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4307811772, "largest_seqno": 4339359193, "table_properties": {"data_size": 11015648, "index_size": 609088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 8893108, "raw_average_key_size": 28, "raw_value_size": 36627730, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 317611, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377925, "oldest_key_time": 0, "file_creation_time": 1732377925, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285293, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:26.995077 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285294.sst) /dev/shm/ToplingTemp/Topling-222928-ZcHJqZ 2024/11/24-00:05:27.549047 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285294.sst): first pass time = 0.55's, 242.301'MB/sec 2024/11/24-00:05:28.887297 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285294.sst): old prealloc_size = 147639500, real_size = 33006000 2024/11/24-00:05:28.887844 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285294.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925219 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.07 avg-zval = 33.52 2024/11/24-00:05:28.888622 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285294.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:05:29.745496 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285294.sst): second pass time = 1.34's, 80.982'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.096'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3371.273'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.13's, 277.492'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34288.017'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.982'MB/sec zip pipeline throughput = 80.982'MB/sec entries = 925219 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.07 avg-zval = 33.52 usrkeys = 925219 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.07 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925219 bytes = 3122624 } seq expand size = 6476533 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5142 value = 3.4923 dict = 1.00 all = 3.6291 } Zip/UnZip{ index = 0.0951 value = 0.2863 dict = 1.00 all = 0.2756 } ---------------------------- total value len = 9.690767 GB avg = 0.164 KB (by entry num) total key len = 1.631353 GB avg = 0.028 KB total ukey len = 1.133972 GB avg = 0.016 KB total ukey num = 0.072355129 Billion total entry num = 0.059162858 Billion write speed all = 0.207298260 MB/sec (with seq num) write speed all = 0.198632492 MB/sec (without seq num) 2024/11/24-00:05:29.799378 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285294: 925219 keys, 36983908 bytes, temperature: kWarm 2024/11/24-00:05:29.799452 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377929799416, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285294, "file_size": 36983908, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4339359548, "table_properties": {"data_size": 31014560, "index_size": 1759936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25906132, "raw_average_key_size": 28, "raw_value_size": 108311698, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925219, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377926, "oldest_key_time": 0, "file_creation_time": 1732377926, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285294, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:29.837200 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285295.sst) /dev/shm/ToplingTemp/Topling-222928-HV5Jj0 2024/11/24-00:05:30.718699 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285295.sst): first pass time = 0.88's, 152.268'MB/sec 2024/11/24-00:05:32.048162 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285295.sst): old prealloc_size = 147639500, real_size = 33423040 2024/11/24-00:05:32.048716 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285295.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925947 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.95 avg-zval = 33.94 2024/11/24-00:05:32.049495 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285295.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:05:32.912045 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285295.sst): second pass time = 1.33's, 81.508'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.299'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3406.574'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.315'MB/sec write SST data time = 0.13's, 291.078'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32258.357'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.508'MB/sec zip pipeline throughput = 81.508'MB/sec entries = 925947 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 33.94 usrkeys = 925947 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 33.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925947 bytes = 3125088 } seq expand size = 6481629 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5110 value = 3.4455 dict = 1.00 all = 3.5884 } Zip/UnZip{ index = 0.0951 value = 0.2902 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 9.805540 GB avg = 0.163 KB (by entry num) total key len = 1.657279 GB avg = 0.028 KB total ukey len = 1.152491 GB avg = 0.016 KB total ukey num = 0.073281076 Billion total entry num = 0.060088805 Billion write speed all = 0.209862179 MB/sec (with seq num) write speed all = 0.201061295 MB/sec (without seq num) 2024/11/24-00:05:32.964040 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285295: 925947 keys, 37403232 bytes, temperature: kWarm 2024/11/24-00:05:32.964113 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377932964075, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285295, "file_size": 37403232, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4339359581, "table_properties": {"data_size": 31429488, "index_size": 1761856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25926516, "raw_average_key_size": 28, "raw_value_size": 108291302, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925947, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377929, "oldest_key_time": 0, "file_creation_time": 1732377929, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285295, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:32.999012 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285296.sst) /dev/shm/ToplingTemp/Topling-222928-nLLuuZ 2024/11/24-00:05:33.863389 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285296.sst): first pass time = 0.86's, 155.284'MB/sec 2024/11/24-00:05:35.238333 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285296.sst): old prealloc_size = 147639500, real_size = 32895744 2024/11/24-00:05:35.238864 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285296.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930231 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.28 avg-zval = 33.21 2024/11/24-00:05:35.240545 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285296.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:05:36.105578 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285296.sst): second pass time = 1.37's, 78.722'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.535'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3512.313'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.13's, 280.163'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30883.792'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.722'MB/sec zip pipeline throughput = 78.722'MB/sec entries = 930231 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.28 avg-zval = 33.21 usrkeys = 930231 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.28 avg-zval = 33.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930231 bytes = 3139552 } seq expand size = 6511617 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5127 value = 3.5015 dict = 1.00 all = 3.6386 } Zip/UnZip{ index = 0.0951 value = 0.2856 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 9.920223 GB avg = 0.163 KB (by entry num) total key len = 1.683325 GB avg = 0.028 KB total ukey len = 1.171095 GB avg = 0.016 KB total ukey num = 0.074211307 Billion total entry num = 0.061019036 Billion write speed all = 0.212426245 MB/sec (with seq num) write speed all = 0.203489637 MB/sec (without seq num) 2024/11/24-00:05:36.156740 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285296: 930231 keys, 36887054 bytes, temperature: kWarm 2024/11/24-00:05:36.156820 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377936156779, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285296, "file_size": 36887054, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4339359706, "table_properties": {"data_size": 30893296, "index_size": 1769728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26046468, "raw_average_key_size": 28, "raw_value_size": 108171402, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930231, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377932, "oldest_key_time": 0, "file_creation_time": 1732377932, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285296, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:36.215365 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285297.sst) /dev/shm/ToplingTemp/Topling-222928-fnASi0 2024/11/24-00:05:37.256219 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285297.sst): first pass time = 1.04's, 128.955'MB/sec 2024/11/24-00:05:38.605544 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285297.sst): old prealloc_size = 147639500, real_size = 33343808 2024/11/24-00:05:38.606101 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285297.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925128 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.08 avg-zval = 33.89 2024/11/24-00:05:38.606853 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285297.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:05:39.462305 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285297.sst): second pass time = 1.35's, 80.322'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.563'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3309.160'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 290.936'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 21218.238'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.322'MB/sec zip pipeline throughput = 80.322'MB/sec entries = 925128 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 33.89 usrkeys = 925128 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 33.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925128 bytes = 3122320 } seq expand size = 6475896 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5140 value = 3.4547 dict = 1.00 all = 3.5962 } Zip/UnZip{ index = 0.0951 value = 0.2895 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 10.035013 GB avg = 0.162 KB (by entry num) total key len = 1.709229 GB avg = 0.028 KB total ukey len = 1.189598 GB avg = 0.016 KB total ukey num = 0.075136435 Billion total entry num = 0.061944164 Billion write speed all = 0.214988714 MB/sec (with seq num) write speed all = 0.205917173 MB/sec (without seq num) 2024/11/24-00:05:39.513511 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285297: 925128 keys, 37321768 bytes, temperature: kWarm 2024/11/24-00:05:39.513586 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377939513550, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285297, "file_size": 37321768, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4339359612, "table_properties": {"data_size": 31352560, "index_size": 1759808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903584, "raw_average_key_size": 28, "raw_value_size": 108314180, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925128, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377936, "oldest_key_time": 0, "file_creation_time": 1732377936, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285297, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:39.556129 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285298.sst) /dev/shm/ToplingTemp/Topling-222928-jha7zY 2024/11/24-00:05:39.596348 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285298.sst): first pass time = 0.04's, 217.819'MB/sec 2024/11/24-00:05:39.678434 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285298.sst): old prealloc_size = 147639500, real_size = 2760944 2024/11/24-00:05:39.678525 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285298.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 59944 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0001 GB avg-key = 16.00 avg-zkey = 1.95 raw-val = 0.0071 GB zip-val = 0.0026 GB avg-val = 118.00 avg-zval = 43.85 2024/11/24-00:05:39.678612 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285298.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0001 GB, waited 0.000 sec, Key+Value = 0.009 GB 2024/11/24-00:05:39.730963 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00/285298.sst): second pass time = 0.08's, 86.860'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.05's, 57.118'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1672.622'MB/sec (index lex order gen) rebuild zvType time = 0.04's, 0.343'MB/sec write SST data time = 0.01's, 340.537'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.072'MB, throughput = 2393.762'MB/sec zip my value time = 0.08's, unzip length = 0.007'GB zip my value throughput = 86.860'MB/sec zip pipeline throughput = 86.860'MB/sec entries = 59944 avg-key = 28.00 avg-zkey = 1.95 avg-val = 118.00 avg-zval = 43.85 usrkeys = 59944 avg-key = 20.00 avg-zkey = 1.95 avg-val = 125.00 avg-zval = 43.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 59944 bytes = 179840 } seq expand size = 419608 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0012 GB value = 0.0071 GB dict = 0.07 MB all = 0.0088 GB } __ZipSize{ index = 0.0001 GB value = 0.0026 GB dict = 0.07 MB all = 0.0030 GB } UnZip/Zip{ index = 10.2531 value = 2.6907 dict = 1.00 all = 2.9177 } Zip/UnZip{ index = 0.0975 value = 0.3717 dict = 1.00 all = 0.3427 } ---------------------------- total value len = 10.042506 GB avg = 0.162 KB (by entry num) total key len = 1.710908 GB avg = 0.028 KB total ukey len = 1.190797 GB avg = 0.016 KB total ukey num = 0.075196379 Billion total entry num = 0.062004108 Billion write speed all = 0.215155547 MB/sec (with seq num) write speed all = 0.206075272 MB/sec (without seq num) 2024/11/24-00:05:39.741467 139957248734784 [compaction_job.cc:1948] [default] [JOB 38296] Generated table #285298: 59944 keys, 2999522 bytes, temperature: kWarm 2024/11/24-00:05:39.741522 139957248734784 EVENT_LOG_v1 {"time_micros": 1732377939741492, "cf_name": "default", "job": 38296, "event": "table_file_creation", "file_number": 285298, "file_size": 2999522, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4314534637, "largest_seqno": 4324569829, "table_properties": {"data_size": 2628832, "index_size": 116928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 1678432, "raw_average_key_size": 28, "raw_value_size": 7073392, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 59944, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377939, "oldest_key_time": 0, "file_creation_time": 1732377939, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285298, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:05:39.752056 139957248734784 [compaction_job.cc:703] job-38296: subcompact[0], size: 0.387900 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38296/att-00 [285287,285288,285289,285290,285291,285292,285293,285294,285295,285296,285297,285298] 2024/11/24-00:05:39.823343 139957248734784 (Original Log Time 2024/11/24-00:05:04.533273) EVENT_LOG_v1 {"time_micros": 1732377904533232, "job": 38296, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [285250], "files_L3": [285261, 285262, 285208, 285209, 285210, 285211, 285212, 285213, 285214, 285215, 285216, 285192, 285274], "score": 0.546475, "input_data_size": 412071619, "oldest_snapshot_seqno": 4339636731} 2024/11/24-00:05:39.823347 139957248734784 (Original Log Time 2024/11/24-00:05:39.822316) [compaction_job.cc:2022] [default] [JOB 38296] Compacted 1@2 + 13@3 files to L3 => 387899929 bytes 2024/11/24-00:05:39.823349 139957248734784 (Original Log Time 2024/11/24-00:05:39.823275) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 12 0 0 0] max score 0.01, MB/sec: 11.7 rd, 11.0 wr, level 3, files in(1, 13) out(12 +0 blob) MB in(27.1, 365.9 +0.0 blob) out(369.9 +0.0 blob), read-write-amplify(28.1) write-amplify(13.6) OK, records in: 9757400, records dropped: 111456 output_compression: Snappy 2024/11/24-00:05:39.823354 139957248734784 (Original Log Time 2024/11/24-00:05:39.823321) EVENT_LOG_v1 {"time_micros": 1732377939823300, "cf": "default", "job": 38296, "event": "compaction_finished", "compaction_time_micros": 35218789, "compaction_time_cpu_micros": 15003556, "output_level": 3, "num_output_files": 12, "total_output_size": 387899929, "num_input_records": 9757400, "num_output_records": 9645944, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 274432363, "file_range_sync_nanos": 0, "file_fsync_nanos": 8049, "file_prepare_write_nanos": 30093, "lsm_state": [0, 0, 0, 12, 0, 0, 0]} 2024/11/24-00:05:39.958946 139957248734784 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732377904,"executesMs":35414,"compactionJobId":38296,"attempt":0,"compactionInputRawBytes":1412277412,"compactionInputZipBytes":412071619,"compactionOutputRawBytes":0,"compactionOutputZipBytes":387899929,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c6357f69d04544cec8f6724d9f9f7494-da8a58784a118a27-00"}}, response = 2024/11/24-00:05:39.959070 139957248734784 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38296/att-00: olev 3, work 35.414 s, result 7.727 ms, install 1.055 ms, input{raw 1.315 GiB zip 392.982 MiB}