2024/11/22-21:05:46.083468 140025062356544 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00: fork to child time = 0.018197 sec 2024/11/22-21:05:46.084281 140025062356544 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00 2024/11/22-21:05:46.092183 140025062356544 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00 2024/11/22-21:05:46.093302 140025062356544 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-21:05:46.093553 140025062356544 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-21:05:46.135250 140025062356544 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-11055/att-00/MANIFEST-206347 2024/11/22-21:05:46.173418 140025062356544 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-11055/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 227791, last_sequence is 3562600757, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 227586 2024/11/22-21:05:46.173426 140025062356544 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-21:05:46.173532 140025062356544 [version_set.cc:5721] Creating manifest 227791 2024/11/22-21:05:46.201558 140025062356544 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-21:05:46.226659 140025062356544 [compaction_job.cc:2437] [default] [JOB 11055] Compacting 1@2 + 16@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/22-21:05:46.226678 140025062356544 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 8330 Base level 2, inputs: [227724(27MB)], [227789(5365KB) 227675(33MB) 227676(35MB) 227693(36MB) 227694(35MB) 227695(35MB) 227696(33MB) 227697(33MB) 227698(34MB) 227699(12MB) 227680(34MB) 227681(35MB) 227682(35MB) 227683(35MB) 227684(35MB) 227770(37MB)] 2024/11/22-21:05:46.235020 140025062356544 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-21:05:46.235053 140025062356544 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-21:05:46.235071 140025062356544 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-21:05:46.235076 140025062356544 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-21:05:46.235090 140025062356544 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-21:05:46.481527 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227792.sst) /dev/shm/ToplingTemp/Topling-290817-8FIoaq 2024/11/22-21:05:47.267046 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227792.sst): first pass time = 0.79's, 170.874'MB/sec 2024/11/22-21:05:48.686346 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227792.sst): old prealloc_size = 147639500, real_size = 32394944 2024/11/22-21:05:48.686900 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227792.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921635 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0304 GB avg-val = 117.63 avg-zval = 32.98 2024/11/22-21:05:48.687725 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227792.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:05:49.744731 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227792.sst): second pass time = 1.42's, 76.449'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 34.577'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3354.473'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.280'MB/sec write SST data time = 0.24's, 154.899'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 34461.754'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.449'MB/sec zip pipeline throughput = 76.449'MB/sec entries = 921635 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.63 avg-zval = 32.98 usrkeys = 921635 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.63 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921635 bytes = 3340944 } seq expand size = 6451445 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4424 value = 3.5663 dict = 1.00 all = 3.6677 } Zip/UnZip{ index = 0.0958 value = 0.2804 dict = 1.00 all = 0.2727 } ---------------------------- total value len = 0.114863 GB avg = 0.125 KB (by entry num) total key len = 0.025806 GB avg = 0.028 KB total ukey len = 0.018433 GB avg = 0.020 KB total ukey num = 0.000921635 Billion total entry num = 0.000921635 Billion write speed all = 0.003196544 MB/sec (with seq num) write speed all = 0.003029000 MB/sec (without seq num) 2024/11/22-21:05:49.779542 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227792: 921635 keys, 36594760 bytes, temperature: kWarm 2024/11/22-21:05:49.779729 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280749779586, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227792, "file_size": 36594760, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3562132268, "table_properties": {"data_size": 30399152, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25805780, "raw_average_key_size": 28, "raw_value_size": 108411982, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921635, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280746, "oldest_key_time": 0, "file_creation_time": 1732280746, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227792, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:05:49.837088 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227793.sst) /dev/shm/ToplingTemp/Topling-290817-TnORar 2024/11/22-21:05:50.703606 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227793.sst): first pass time = 0.87's, 154.905'MB/sec 2024/11/22-21:05:52.091812 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227793.sst): old prealloc_size = 147639500, real_size = 33652672 2024/11/22-21:05:52.092372 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227793.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924213 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0317 GB avg-val = 117.22 avg-zval = 34.26 2024/11/22-21:05:52.093136 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227793.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:05:53.077919 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227793.sst): second pass time = 1.39's, 78.083'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 38.276'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3271.729'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.17's, 220.725'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 27072.492'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 78.083'MB/sec zip pipeline throughput = 78.083'MB/sec entries = 924213 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.22 avg-zval = 34.26 usrkeys = 924213 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.22 avg-zval = 34.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924213 bytes = 3234768 } seq expand size = 6469491 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5112 value = 3.4217 dict = 1.00 all = 3.5560 } Zip/UnZip{ index = 0.0951 value = 0.2923 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 0.229673 GB avg = 0.124 KB (by entry num) total key len = 0.051684 GB avg = 0.028 KB total ukey len = 0.036917 GB avg = 0.020 KB total ukey num = 0.001845848 Billion total entry num = 0.001845848 Billion write speed all = 0.006393016 MB/sec (with seq num) write speed all = 0.006057483 MB/sec (without seq num) 2024/11/22-21:05:53.111546 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227793: 924213 keys, 37744122 bytes, temperature: kWarm 2024/11/22-21:05:53.111609 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280753111578, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227793, "file_size": 37744122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3378289821, "largest_seqno": 3562135005, "table_properties": {"data_size": 31662896, "index_size": 1758528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25877964, "raw_average_key_size": 28, "raw_value_size": 108339846, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924213, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280749, "oldest_key_time": 0, "file_creation_time": 1732280749, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227793, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:05:53.148858 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227794.sst) /dev/shm/ToplingTemp/Topling-290817-mFwR5r 2024/11/22-21:05:54.124433 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227794.sst): first pass time = 0.98's, 137.584'MB/sec 2024/11/22-21:05:55.484713 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227794.sst): old prealloc_size = 147639500, real_size = 33580064 2024/11/22-21:05:55.485302 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227794.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924203 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.23 avg-zval = 34.17 2024/11/22-21:05:55.486048 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227794.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:05:56.421694 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227794.sst): second pass time = 1.36's, 79.683'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.206'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3116.624'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 291.605'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 37121.633'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.683'MB/sec zip pipeline throughput = 79.683'MB/sec entries = 924203 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.23 avg-zval = 34.17 usrkeys = 924203 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.23 avg-zval = 34.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924203 bytes = 3234720 } seq expand size = 6469421 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4446 value = 3.4308 dict = 1.00 all = 3.5628 } Zip/UnZip{ index = 0.0957 value = 0.2915 dict = 1.00 all = 0.2807 } ---------------------------- total value len = 0.344482 GB avg = 0.124 KB (by entry num) total key len = 0.077561 GB avg = 0.028 KB total ukey len = 0.055401 GB avg = 0.020 KB total ukey num = 0.002770051 Billion total entry num = 0.002770051 Billion write speed all = 0.009588998 MB/sec (with seq num) write speed all = 0.009085505 MB/sec (without seq num) 2024/11/22-21:05:56.454341 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227794: 924203 keys, 37671494 bytes, temperature: kWarm 2024/11/22-21:05:56.454402 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280756454370, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227794, "file_size": 37671494, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774735, "largest_seqno": 3562134743, "table_properties": {"data_size": 31579088, "index_size": 1769728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25877684, "raw_average_key_size": 28, "raw_value_size": 108340094, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924203, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280753, "oldest_key_time": 0, "file_creation_time": 1732280753, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227794, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:05:56.489203 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227795.sst) /dev/shm/ToplingTemp/Topling-290817-nt4vIp 2024/11/22-21:05:57.421034 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227795.sst): first pass time = 0.93's, 144.043'MB/sec 2024/11/22-21:05:58.841769 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227795.sst): old prealloc_size = 147639500, real_size = 32960704 2024/11/22-21:05:58.842470 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227795.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924177 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.23 avg-zval = 33.51 2024/11/22-21:05:58.843296 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227795.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:06:00.214751 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227795.sst): second pass time = 1.42's, 76.343'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.37's, 26.987'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2650.720'MB/sec (index lex order gen) rebuild zvType time = 1.15's, 0.201'MB/sec write SST data time = 0.22's, 166.828'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 26950.901'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.343'MB/sec zip pipeline throughput = 76.343'MB/sec entries = 924177 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.23 avg-zval = 33.51 usrkeys = 924177 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.23 avg-zval = 33.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924177 bytes = 3234640 } seq expand size = 6469239 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5142 value = 3.4981 dict = 1.00 all = 3.6224 } Zip/UnZip{ index = 0.0951 value = 0.2859 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 0.459292 GB avg = 0.124 KB (by entry num) total key len = 0.103438 GB avg = 0.028 KB total ukey len = 0.073885 GB avg = 0.020 KB total ukey num = 0.003694228 Billion total entry num = 0.003694228 Billion write speed all = 0.012784361 MB/sec (with seq num) write speed all = 0.012112944 MB/sec (without seq num) 2024/11/22-21:06:00.246250 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227795: 924177 keys, 37052018 bytes, temperature: kWarm 2024/11/22-21:06:00.246311 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280760246280, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227795, "file_size": 37052018, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3562134758, "table_properties": {"data_size": 30971504, "index_size": 1757952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876956, "raw_average_key_size": 28, "raw_value_size": 108340830, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924177, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280756, "oldest_key_time": 0, "file_creation_time": 1732280756, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227795, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:00.288030 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227796.sst) /dev/shm/ToplingTemp/Topling-290817-mkFuAp 2024/11/22-21:06:01.156386 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227796.sst): first pass time = 0.87's, 154.573'MB/sec 2024/11/22-21:06:02.486389 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227796.sst): old prealloc_size = 147639500, real_size = 31488288 2024/11/22-21:06:02.486969 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227796.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924188 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0295 GB avg-val = 117.23 avg-zval = 31.91 2024/11/22-21:06:02.487743 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227796.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:06:03.342958 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227796.sst): second pass time = 1.33's, 81.497'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.538'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.583'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 275.122'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33803.737'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.497'MB/sec zip pipeline throughput = 81.497'MB/sec entries = 924188 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.23 avg-zval = 31.91 usrkeys = 924188 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.23 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924188 bytes = 3234672 } seq expand size = 6469316 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4444 value = 3.6741 dict = 1.00 all = 3.7723 } Zip/UnZip{ index = 0.0957 value = 0.2722 dict = 1.00 all = 0.2651 } ---------------------------- total value len = 0.574102 GB avg = 0.124 KB (by entry num) total key len = 0.129316 GB avg = 0.028 KB total ukey len = 0.092368 GB avg = 0.020 KB total ukey num = 0.004618416 Billion total entry num = 0.004618416 Billion write speed all = 0.015979416 MB/sec (with seq num) write speed all = 0.015140090 MB/sec (without seq num) 2024/11/22-21:06:03.374906 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227796: 924188 keys, 35579670 bytes, temperature: kWarm 2024/11/22-21:06:03.374969 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280763374937, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227796, "file_size": 35579670, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774766, "largest_seqno": 3562135111, "table_properties": {"data_size": 29487312, "index_size": 1769728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25877264, "raw_average_key_size": 28, "raw_value_size": 108340536, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924188, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280760, "oldest_key_time": 0, "file_creation_time": 1732280760, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227796, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:03.414071 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227797.sst) /dev/shm/ToplingTemp/Topling-290817-zikC3n 2024/11/22-21:06:04.326255 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227797.sst): first pass time = 0.91's, 147.146'MB/sec 2024/11/22-21:06:05.646203 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227797.sst): old prealloc_size = 147639500, real_size = 31216704 2024/11/22-21:06:05.646758 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227797.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920805 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.76 avg-zval = 31.74 2024/11/22-21:06:05.647536 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227797.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:06:06.508336 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227797.sst): second pass time = 1.32's, 82.253'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.943'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3314.333'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.314'MB/sec write SST data time = 0.13's, 273.045'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 31956.847'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.253'MB/sec zip pipeline throughput = 82.253'MB/sec entries = 920805 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.76 avg-zval = 31.74 usrkeys = 920805 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.76 avg-zval = 31.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920805 bytes = 3222832 } seq expand size = 6445635 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4439 value = 3.7106 dict = 1.00 all = 3.8025 } Zip/UnZip{ index = 0.0957 value = 0.2695 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 0.688983 GB avg = 0.124 KB (by entry num) total key len = 0.155098 GB avg = 0.028 KB total ukey len = 0.110784 GB avg = 0.020 KB total ukey num = 0.005539221 Billion total entry num = 0.005539221 Billion write speed all = 0.019173465 MB/sec (with seq num) write speed all = 0.018166869 MB/sec (without seq num) 2024/11/22-21:06:06.539028 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227797: 920805 keys, 35297432 bytes, temperature: kWarm 2024/11/22-21:06:06.539092 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280766539060, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227797, "file_size": 35297432, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3386023460, "largest_seqno": 3562134791, "table_properties": {"data_size": 29222960, "index_size": 1763328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25782540, "raw_average_key_size": 28, "raw_value_size": 108435310, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920805, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280763, "oldest_key_time": 0, "file_creation_time": 1732280763, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227797, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:06.577425 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227798.sst) /dev/shm/ToplingTemp/Topling-290817-wTrhLn 2024/11/22-21:06:07.417198 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227798.sst): first pass time = 0.84's, 159.834'MB/sec 2024/11/22-21:06:08.783989 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227798.sst): old prealloc_size = 147639500, real_size = 32462896 2024/11/22-21:06:08.784531 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227798.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922732 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.46 avg-zval = 33.03 2024/11/22-21:06:08.785318 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227798.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:06:09.689059 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227798.sst): second pass time = 1.37's, 79.334'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.384'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3372.706'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.13's, 283.109'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 23184.199'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.334'MB/sec zip pipeline throughput = 79.334'MB/sec entries = 922732 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 33.03 usrkeys = 922732 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 33.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922732 bytes = 3229584 } seq expand size = 6459124 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5150 value = 3.5562 dict = 1.00 all = 3.6721 } Zip/UnZip{ index = 0.0951 value = 0.2812 dict = 1.00 all = 0.2723 } ---------------------------- total value len = 0.803824 GB avg = 0.124 KB (by entry num) total key len = 0.180935 GB avg = 0.028 KB total ukey len = 0.129239 GB avg = 0.020 KB total ukey num = 0.006461953 Billion total entry num = 0.006461953 Billion write speed all = 0.022367350 MB/sec (with seq num) write speed all = 0.021193159 MB/sec (without seq num) 2024/11/22-21:06:09.721533 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227798: 922732 keys, 36550482 bytes, temperature: kWarm 2024/11/22-21:06:09.721597 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280769721564, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227798, "file_size": 36550482, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774786, "largest_seqno": 3562134917, "table_properties": {"data_size": 30476960, "index_size": 1755072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25836496, "raw_average_key_size": 28, "raw_value_size": 108381268, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922732, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280766, "oldest_key_time": 0, "file_creation_time": 1732280766, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227798, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:09.756808 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227799.sst) /dev/shm/ToplingTemp/Topling-290817-4JuUcp 2024/11/22-21:06:10.626152 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227799.sst): first pass time = 0.87's, 154.397'MB/sec 2024/11/22-21:06:11.976113 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227799.sst): old prealloc_size = 147639500, real_size = 33450528 2024/11/22-21:06:11.976684 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227799.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924288 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.21 avg-zval = 34.04 2024/11/22-21:06:11.977470 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227799.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:06:12.919856 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227799.sst): second pass time = 1.35's, 80.323'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.781'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3226.357'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.13's, 291.294'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32962.584'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.323'MB/sec zip pipeline throughput = 80.323'MB/sec entries = 924288 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.21 avg-zval = 34.04 usrkeys = 924288 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.21 avg-zval = 34.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924288 bytes = 3235024 } seq expand size = 6470016 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5151 value = 3.4435 dict = 1.00 all = 3.5751 } Zip/UnZip{ index = 0.0951 value = 0.2904 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 0.918631 GB avg = 0.124 KB (by entry num) total key len = 0.206815 GB avg = 0.028 KB total ukey len = 0.147725 GB avg = 0.020 KB total ukey num = 0.007386241 Billion total entry num = 0.007386241 Billion write speed all = 0.025560994 MB/sec (with seq num) write speed all = 0.024218951 MB/sec (without seq num) 2024/11/22-21:06:12.954859 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227799: 924288 keys, 37542136 bytes, temperature: kWarm 2024/11/22-21:06:12.954923 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280772954891, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227799, "file_size": 37542136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774802, "largest_seqno": 3562134809, "table_properties": {"data_size": 31461264, "index_size": 1758016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25880064, "raw_average_key_size": 28, "raw_value_size": 108337756, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924288, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280769, "oldest_key_time": 0, "file_creation_time": 1732280769, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227799, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:12.989613 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227800.sst) /dev/shm/ToplingTemp/Topling-290817-JGgG4r 2024/11/22-21:06:13.945173 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227800.sst): first pass time = 0.96's, 140.467'MB/sec 2024/11/22-21:06:15.350332 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227800.sst): old prealloc_size = 147639500, real_size = 35110272 2024/11/22-21:06:15.350861 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227800.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927998 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.63 avg-zval = 35.66 2024/11/22-21:06:15.351323 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227800.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:06:16.157392 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227800.sst): second pass time = 1.40's, 77.096'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 46.610'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3535.834'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.343'MB/sec write SST data time = 0.13's, 290.560'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30168.813'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.096'MB/sec zip pipeline throughput = 77.096'MB/sec entries = 927998 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.63 avg-zval = 35.66 usrkeys = 927998 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.12 avg-zval = 35.66 TagRS{ kinds = RS_Key0_TagN bytes = 145080 } TagArray{ size = 462747 bytes = 1503936 } seq expand size = 3239229 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.3760 value = 3.2710 dict = 1.00 all = 3.5681 } Zip/UnZip{ index = 0.0964 value = 0.3057 dict = 1.00 all = 0.2803 } ---------------------------- total value len = 1.030104 GB avg = 0.124 KB (by entry num) total key len = 0.232799 GB avg = 0.028 KB total ukey len = 0.166285 GB avg = 0.020 KB total ukey num = 0.008314239 Billion total entry num = 0.008314239 Billion write speed all = 0.028680792 MB/sec (with seq num) write speed all = 0.027170247 MB/sec (without seq num) 2024/11/22-21:06:16.194597 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227800: 927998 keys, 37616555 bytes, temperature: kWarm 2024/11/22-21:06:16.194663 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280776194630, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227800, "file_size": 37616555, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3562135084, "table_properties": {"data_size": 33089328, "index_size": 1788736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25983944, "raw_average_key_size": 28, "raw_value_size": 108233852, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927998, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280772, "oldest_key_time": 0, "file_creation_time": 1732280772, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227800, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:16.231523 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227801.sst) /dev/shm/ToplingTemp/Topling-290817-wzxJhr 2024/11/22-21:06:17.136189 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227801.sst): first pass time = 0.90's, 148.369'MB/sec 2024/11/22-21:06:18.486847 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227801.sst): old prealloc_size = 147639500, real_size = 35476080 2024/11/22-21:06:18.487468 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227801.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080301 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1040 GB zip-val = 0.0332 GB avg-val = 96.24 avg-zval = 30.69 2024/11/22-21:06:18.487718 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227801.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:06:19.148609 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227801.sst): second pass time = 1.35's, 77.018'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.66's, 55.855'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3452.957'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.521'MB/sec write SST data time = 0.14's, 259.581'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.026'MB, throughput = 29242.168'MB/sec zip my value time = 1.35's, unzip length = 0.104'GB zip my value throughput = 77.018'MB/sec zip pipeline throughput = 77.018'MB/sec entries = 1080301 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.24 avg-zval = 30.69 usrkeys = 1080301 avg-key = 20.00 avg-zkey = 1.90 avg-val = 97.26 avg-zval = 30.69 TagRS{ kinds = RS_Key0_TagN bytes = 168880 } TagArray{ size = 157149 bytes = 550032 } seq expand size = 1100043 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0332 GB dict = 1.03 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5160 value = 3.1362 dict = 1.00 all = 3.6313 } Zip/UnZip{ index = 0.0951 value = 0.3189 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 1.135174 GB avg = 0.121 KB (by entry num) total key len = 0.263047 GB avg = 0.028 KB total ukey len = 0.187891 GB avg = 0.020 KB total ukey num = 0.009394540 Billion total entry num = 0.009394540 Billion write speed all = 0.031751732 MB/sec (with seq num) write speed all = 0.030045032 MB/sec (without seq num) 2024/11/22-21:06:19.180588 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227801: 1080301 keys, 36961545 bytes, temperature: kWarm 2024/11/22-21:06:19.180660 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280779180620, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227801, "file_size": 36961545, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3411649181, "table_properties": {"data_size": 33151200, "index_size": 2054592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30248428, "raw_average_key_size": 28, "raw_value_size": 103969391, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080301, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280776, "oldest_key_time": 0, "file_creation_time": 1732280776, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227801, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:19.217456 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227802.sst) /dev/shm/ToplingTemp/Topling-290817-OOb3Wp 2024/11/22-21:06:20.085608 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227802.sst): first pass time = 0.87's, 154.608'MB/sec 2024/11/22-21:06:21.425444 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227802.sst): old prealloc_size = 147639500, real_size = 33869792 2024/11/22-21:06:21.425997 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227802.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919783 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0319 GB avg-val = 117.92 avg-zval = 34.67 2024/11/22-21:06:21.427764 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227802.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:06:22.353069 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227802.sst): second pass time = 1.34's, 80.993'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.911'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3278.349'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.13's, 291.394'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34452.230'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.993'MB/sec zip pipeline throughput = 80.993'MB/sec entries = 919783 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.92 avg-zval = 34.67 usrkeys = 919783 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.92 avg-zval = 34.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919783 bytes = 3219248 } seq expand size = 6438481 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5140 value = 3.4012 dict = 1.00 all = 3.5367 } Zip/UnZip{ index = 0.0951 value = 0.2940 dict = 1.00 all = 0.2827 } ---------------------------- total value len = 1.250076 GB avg = 0.121 KB (by entry num) total key len = 0.288801 GB avg = 0.028 KB total ukey len = 0.206286 GB avg = 0.020 KB total ukey num = 0.010314323 Billion total entry num = 0.010314323 Billion write speed all = 0.034943306 MB/sec (with seq num) write speed all = 0.033069646 MB/sec (without seq num) 2024/11/22-21:06:22.386669 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227802: 919783 keys, 37949478 bytes, temperature: kWarm 2024/11/22-21:06:22.386731 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280782386700, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227802, "file_size": 37949478, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411649182, "largest_seqno": 3562134192, "table_properties": {"data_size": 31890000, "index_size": 1749632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25753924, "raw_average_key_size": 28, "raw_value_size": 108463874, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919783, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280779, "oldest_key_time": 0, "file_creation_time": 1732280779, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227802, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:22.421095 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227803.sst) /dev/shm/ToplingTemp/Topling-290817-407oAr 2024/11/22-21:06:23.314621 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227803.sst): first pass time = 0.89's, 150.218'MB/sec 2024/11/22-21:06:24.659694 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227803.sst): old prealloc_size = 147639500, real_size = 34469536 2024/11/22-21:06:24.660254 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227803.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929183 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.45 avg-zval = 34.93 2024/11/22-21:06:24.662063 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227803.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:06:25.572737 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227803.sst): second pass time = 1.34's, 80.480'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.376'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3289.468'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.13's, 296.145'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33289.976'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.480'MB/sec zip pipeline throughput = 80.480'MB/sec entries = 929183 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.45 avg-zval = 34.93 usrkeys = 929183 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.45 avg-zval = 34.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929183 bytes = 3368304 } seq expand size = 6504281 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.4423 value = 3.3336 dict = 1.00 all = 3.4690 } Zip/UnZip{ index = 0.0958 value = 0.3000 dict = 1.00 all = 0.2883 } ---------------------------- total value len = 1.364781 GB avg = 0.121 KB (by entry num) total key len = 0.314818 GB avg = 0.028 KB total ukey len = 0.224870 GB avg = 0.020 KB total ukey num = 0.011243506 Billion total entry num = 0.011243506 Billion write speed all = 0.038135895 MB/sec (with seq num) write speed all = 0.036093593 MB/sec (without seq num) 2024/11/22-21:06:25.609479 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227803: 929183 keys, 38690436 bytes, temperature: kWarm 2024/11/22-21:06:25.609538 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280785609508, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227803, "file_size": 38690436, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3562134362, "table_properties": {"data_size": 32457424, "index_size": 1779648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26017124, "raw_average_key_size": 28, "raw_value_size": 108200650, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929183, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280782, "oldest_key_time": 0, "file_creation_time": 1732280782, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227803, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:25.648413 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227804.sst) /dev/shm/ToplingTemp/Topling-290817-yIfgVp 2024/11/22-21:06:26.536736 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227804.sst): first pass time = 0.89's, 151.099'MB/sec 2024/11/22-21:06:27.880800 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227804.sst): old prealloc_size = 147639500, real_size = 33794144 2024/11/22-21:06:27.881353 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227804.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929560 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0318 GB avg-val = 116.39 avg-zval = 34.19 2024/11/22-21:06:27.883198 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227804.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:06:28.814041 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227804.sst): second pass time = 1.34's, 80.560'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.612'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3462.084'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.289'MB/sec write SST data time = 0.13's, 293.700'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33776.526'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.560'MB/sec zip pipeline throughput = 80.560'MB/sec entries = 929560 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.39 avg-zval = 34.19 usrkeys = 929560 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.39 avg-zval = 34.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929560 bytes = 3253472 } seq expand size = 6506920 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4447 value = 3.4042 dict = 1.00 all = 3.5413 } Zip/UnZip{ index = 0.0957 value = 0.2938 dict = 1.00 all = 0.2824 } ---------------------------- total value len = 1.479478 GB avg = 0.122 KB (by entry num) total key len = 0.340846 GB avg = 0.028 KB total ukey len = 0.243461 GB avg = 0.020 KB total ukey num = 0.012173066 Billion total entry num = 0.012173066 Billion write speed all = 0.041328059 MB/sec (with seq num) write speed all = 0.039117072 MB/sec (without seq num) 2024/11/22-21:06:28.846958 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227804: 929560 keys, 37900536 bytes, temperature: kWarm 2024/11/22-21:06:28.847021 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280788846988, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227804, "file_size": 37900536, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3341636974, "largest_seqno": 3562132196, "table_properties": {"data_size": 31781584, "index_size": 1779968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26027680, "raw_average_key_size": 28, "raw_value_size": 108190160, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929560, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280785, "oldest_key_time": 0, "file_creation_time": 1732280785, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227804, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:28.884878 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227805.sst) /dev/shm/ToplingTemp/Topling-290817-eYtPEn 2024/11/22-21:06:29.766112 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227805.sst): first pass time = 0.88's, 152.314'MB/sec 2024/11/22-21:06:31.087638 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227805.sst): old prealloc_size = 147639500, real_size = 33444496 2024/11/22-21:06:31.088202 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227805.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924144 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.23 avg-zval = 34.04 2024/11/22-21:06:31.090011 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227805.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-21:06:32.018117 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227805.sst): second pass time = 1.32's, 82.060'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.341'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3251.118'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 292.809'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33275.176'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.060'MB/sec zip pipeline throughput = 82.060'MB/sec entries = 924144 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.23 avg-zval = 34.04 usrkeys = 924144 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.23 avg-zval = 34.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924144 bytes = 3234512 } seq expand size = 6469008 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5123 value = 3.4443 dict = 1.00 all = 3.5757 } Zip/UnZip{ index = 0.0951 value = 0.2903 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 1.594289 GB avg = 0.122 KB (by entry num) total key len = 0.366722 GB avg = 0.028 KB total ukey len = 0.261944 GB avg = 0.020 KB total ukey num = 0.013097210 Billion total entry num = 0.013097210 Billion write speed all = 0.044518928 MB/sec (with seq num) write speed all = 0.042140262 MB/sec (without seq num) 2024/11/22-21:06:32.050635 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227805: 924144 keys, 37535918 bytes, temperature: kWarm 2024/11/22-21:06:32.050698 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280792050666, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227805, "file_size": 37535918, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3562134644, "table_properties": {"data_size": 31455040, "index_size": 1758208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876032, "raw_average_key_size": 28, "raw_value_size": 108341728, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924144, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280788, "oldest_key_time": 0, "file_creation_time": 1732280788, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227805, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:32.091121 140025062356544 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227806.sst) /dev/shm/ToplingTemp/Topling-290817-dFF2ro 2024/11/22-21:06:32.435174 140025062356544 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227806.sst): first pass time = 0.34's, 243.182'MB/sec 2024/11/22-21:06:33.316111 140025062356544 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227806.sst): old prealloc_size = 147639500, real_size = 22793312 2024/11/22-21:06:33.316461 140025062356544 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227806.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 577124 prefix = 4 raw-key = 0.0092 GB zip-key = 0.0011 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0675 GB zip-val = 0.0216 GB avg-val = 116.96 avg-zval = 37.34 2024/11/22-21:06:33.316959 140025062356544 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227806.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0014 GB, waited 0.000 sec, Key+Value = 0.084 GB 2024/11/22-21:06:33.854434 140025062356544 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00/227806.sst): second pass time = 0.88's, 76.705'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.54's, 46.826'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3320.838'MB/sec (index lex order gen) rebuild zvType time = 0.46's, 0.315'MB/sec write SST data time = 0.08's, 314.856'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.678'MB, throughput = 18864.788'MB/sec zip my value time = 0.88's, unzip length = 0.067'GB zip my value throughput = 76.705'MB/sec zip pipeline throughput = 76.705'MB/sec entries = 577124 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 37.34 usrkeys = 577124 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 37.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 577124 bytes = 1875664 } seq expand size = 4039868 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0115 GB value = 0.0675 GB dict = 0.68 MB all = 0.0837 GB } __ZipSize{ index = 0.0011 GB value = 0.0216 GB dict = 0.68 MB all = 0.0252 GB } UnZip/Zip{ index = 10.5094 value = 3.1321 dict = 1.00 all = 3.3188 } Zip/UnZip{ index = 0.0952 value = 0.3193 dict = 1.00 all = 0.3013 } ---------------------------- total value len = 1.665826 GB avg = 0.122 KB (by entry num) total key len = 0.382881 GB avg = 0.028 KB total ukey len = 0.273487 GB avg = 0.020 KB total ukey num = 0.013674334 Billion total entry num = 0.013674334 Billion write speed all = 0.046507889 MB/sec (with seq num) write speed all = 0.044024512 MB/sec (without seq num) 2024/11/22-21:06:33.879407 140025062356544 [compaction_job.cc:1948] [default] [JOB 11055] Generated table #227806: 577124 keys, 25206991 bytes, temperature: kWarm 2024/11/22-21:06:33.879465 140025062356544 EVENT_LOG_v1 {"time_micros": 1732280793879435, "cf_name": "default", "job": 11055, "event": "table_file_creation", "file_number": 227806, "file_size": 25206991, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3515630317, "largest_seqno": 3562134548, "table_properties": {"data_size": 21550544, "index_size": 1098304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 16159472, "raw_average_key_size": 28, "raw_value_size": 67497652, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 577124, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732280792, "oldest_key_time": 0, "file_creation_time": 1732280792, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 227806, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-21:06:33.900039 140025062356544 [compaction_job.cc:703] job-11055: subcompact[0], size: 0.545894 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-11055/att-00 [227792,227793,227794,227795,227796,227797,227798,227799,227800,227801,227802,227803,227804,227805,227806] 2024/11/22-21:06:33.998014 140025062356544 (Original Log Time 2024/11/22-21:05:46.226726) EVENT_LOG_v1 {"time_micros": 1732280746226690, "job": 11055, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [227724], "files_L3": [227789, 227675, 227676, 227693, 227694, 227695, 227696, 227697, 227698, 227699, 227680, 227681, 227682, 227683, 227684, 227770], "score": 0.994579, "input_data_size": 563886791, "oldest_snapshot_seqno": 3562600697} 2024/11/22-21:06:33.998018 140025062356544 (Original Log Time 2024/11/22-21:06:33.996875) [compaction_job.cc:2022] [default] [JOB 11055] Compacted 1@2 + 16@3 files to L3 => 545893573 bytes 2024/11/22-21:06:33.998020 140025062356544 (Original Log Time 2024/11/22-21:06:33.997559) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.8 rd, 11.5 wr, level 3, files in(1, 16) out(15 +0 blob) MB in(27.2, 510.6 +0.0 blob) out(520.6 +0.0 blob), read-write-amplify(38.9) write-amplify(19.2) OK, records in: 13761004, records dropped: 86670 output_compression: Snappy 2024/11/22-21:06:33.998026 140025062356544 (Original Log Time 2024/11/22-21:06:33.997992) EVENT_LOG_v1 {"time_micros": 1732280793997582, "cf": "default", "job": 11055, "event": "compaction_finished", "compaction_time_micros": 47673322, "compaction_time_cpu_micros": 21557178, "output_level": 3, "num_output_files": 15, "total_output_size": 545893573, "num_input_records": 13761004, "num_output_records": 13674334, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 351381886, "file_range_sync_nanos": 0, "file_fsync_nanos": 14864, "file_prepare_write_nanos": 38829, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/22-21:06:34.138507 140025062356544 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732280746,"executesMs":47906,"compactionJobId":11055,"attempt":0,"compactionInputRawBytes":1975013421,"compactionInputZipBytes":563886791,"compactionOutputRawBytes":0,"compactionOutputZipBytes":545893573,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c8b66fbdf2ff05f37d703dddd8040280-6e00ed266dca808f-00"}}, response = 2024/11/22-21:06:34.138577 140025062356544 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-11055/att-00: olev 3, work 47.907 s, result 6.350 ms, install 1.173 ms, input{raw 1.839 GiB zip 537.764 MiB}