2024/11/22-22:53:59.765285 140024844146240 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00: fork to child time = 0.026019 sec 2024/11/22-22:53:59.766357 140024844146240 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00 2024/11/22-22:53:59.774719 140024844146240 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00 2024/11/22-22:53:59.776144 140024844146240 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-22:53:59.776465 140024844146240 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-22:53:59.826172 140024844146240 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13283/att-00/MANIFEST-206347 2024/11/22-22:53:59.840008 140024844146240 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13283/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 232408, last_sequence is 3624136375, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 232256 2024/11/22-22:53:59.840018 140024844146240 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-22:53:59.840162 140024844146240 [version_set.cc:5721] Creating manifest 232408 2024/11/22-22:53:59.871988 140024844146240 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-22:53:59.945788 140024844146240 [compaction_job.cc:2437] [default] [JOB 13283] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-22:53:59.945817 140024844146240 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 9988 Base level 2, inputs: [232405(27MB)], [232334(35MB) 232335(35MB) 232336(35MB) 232337(34MB) 232338(35MB) 232339(35MB) 232340(35MB) 232341(35MB) 232342(38MB) 232343(35MB) 232344(35MB) 232345(35MB) 232346(35MB) 232347(36MB) 232360(36MB)] 2024/11/22-22:53:59.953789 140024844146240 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:53:59.953842 140024844146240 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:53:59.953871 140024844146240 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:53:59.953879 140024844146240 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:53:59.953935 140024844146240 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:54:00.522095 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232409.sst) /dev/shm/ToplingTemp/Topling-322921-wuMlYW 2024/11/22-22:54:01.230712 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232409.sst): first pass time = 0.71's, 189.423'MB/sec 2024/11/22-22:54:03.531641 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232409.sst): old prealloc_size = 147639500, real_size = 32807520 2024/11/22-22:54:03.532292 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232409.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924943 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.11 avg-zval = 33.32 2024/11/22-22:54:03.533125 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232409.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:04.700205 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232409.sst): second pass time = 2.30's, 47.108'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.17's, 31.478'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2808.160'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.252'MB/sec write SST data time = 0.25's, 146.509'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 23800.409'MB/sec zip my value time = 2.30's, unzip length = 0.108'GB zip my value throughput = 47.108'MB/sec zip pipeline throughput = 104.354'MB/sec entries = 924943 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.11 avg-zval = 33.32 usrkeys = 924943 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.11 avg-zval = 33.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924943 bytes = 3121696 } seq expand size = 6474601 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5145 value = 3.5150 dict = 1.00 all = 3.6487 } Zip/UnZip{ index = 0.0951 value = 0.2845 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 0.114794 GB avg = 0.124 KB (by entry num) total key len = 0.025898 GB avg = 0.028 KB total ukey len = 0.018499 GB avg = 0.020 KB total ukey num = 0.000924943 Billion total entry num = 0.000924943 Billion write speed all = 0.002785899 MB/sec (with seq num) write speed all = 0.002639378 MB/sec (without seq num) 2024/11/22-22:54:04.734864 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232409: 924943 keys, 36784846 bytes, temperature: kWarm 2024/11/22-22:54:04.734953 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287244734899, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232409, "file_size": 36784846, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3513012344, "largest_seqno": 3624093876, "table_properties": {"data_size": 30816720, "index_size": 1759360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25898404, "raw_average_key_size": 28, "raw_value_size": 108319354, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924943, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287240, "oldest_key_time": 0, "file_creation_time": 1732287240, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232409, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:04.771523 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232410.sst) /dev/shm/ToplingTemp/Topling-322921-nCtD3V 2024/11/22-22:54:05.856876 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232410.sst): first pass time = 1.09's, 123.668'MB/sec 2024/11/22-22:54:07.298365 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232410.sst): old prealloc_size = 147639500, real_size = 33523024 2024/11/22-22:54:07.298938 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232410.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924714 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.15 avg-zval = 34.10 2024/11/22-22:54:07.299705 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232410.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:08.756524 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232410.sst): second pass time = 1.44's, 75.186'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.46's, 25.797'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3201.840'MB/sec (index lex order gen) rebuild zvType time = 1.13's, 0.205'MB/sec write SST data time = 0.33's, 113.273'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35954.647'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.186'MB/sec zip pipeline throughput = 75.186'MB/sec entries = 924714 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 34.10 usrkeys = 924714 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 34.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924714 bytes = 3236512 } seq expand size = 6472998 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5119 value = 3.4354 dict = 1.00 all = 3.5682 } Zip/UnZip{ index = 0.0951 value = 0.2911 dict = 1.00 all = 0.2803 } ---------------------------- total value len = 0.229593 GB avg = 0.124 KB (by entry num) total key len = 0.051790 GB avg = 0.028 KB total ukey len = 0.036993 GB avg = 0.020 KB total ukey num = 0.001849657 Billion total entry num = 0.001849657 Billion write speed all = 0.005571319 MB/sec (with seq num) write speed all = 0.005278337 MB/sec (without seq num) 2024/11/22-22:54:08.798071 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232410: 924714 keys, 37615496 bytes, temperature: kWarm 2024/11/22-22:54:08.798150 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287248798107, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232410, "file_size": 37615496, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3624093896, "table_properties": {"data_size": 31532288, "index_size": 1759360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25891992, "raw_average_key_size": 28, "raw_value_size": 108325792, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924714, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287244, "oldest_key_time": 0, "file_creation_time": 1732287244, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232410, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:08.872871 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232411.sst) /dev/shm/ToplingTemp/Topling-322921-B3TadX 2024/11/22-22:54:10.259190 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232411.sst): first pass time = 1.39's, 96.820'MB/sec 2024/11/22-22:54:11.922338 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232411.sst): old prealloc_size = 147639500, real_size = 33409680 2024/11/22-22:54:11.922976 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232411.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924986 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.10 avg-zval = 33.97 2024/11/22-22:54:11.923744 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232411.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:13.576701 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232411.sst): second pass time = 1.66's, 65.156'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.65's, 22.599'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2858.541'MB/sec (index lex order gen) rebuild zvType time = 1.39's, 0.166'MB/sec write SST data time = 0.26's, 142.818'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 24511.892'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.156'MB/sec zip pipeline throughput = 65.156'MB/sec entries = 924986 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 33.97 usrkeys = 924986 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 33.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924986 bytes = 3121840 } seq expand size = 6474902 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5139 value = 3.4476 dict = 1.00 all = 3.5899 } Zip/UnZip{ index = 0.0951 value = 0.2901 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 0.344386 GB avg = 0.124 KB (by entry num) total key len = 0.077690 GB avg = 0.028 KB total ukey len = 0.055493 GB avg = 0.020 KB total ukey num = 0.002774643 Billion total entry num = 0.002774643 Billion write speed all = 0.008356205 MB/sec (with seq num) write speed all = 0.007916748 MB/sec (without seq num) 2024/11/22-22:54:13.626398 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232411: 924986 keys, 37387210 bytes, temperature: kWarm 2024/11/22-22:54:13.626477 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287253626435, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232411, "file_size": 37387210, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3543969197, "largest_seqno": 3624093915, "table_properties": {"data_size": 31418688, "index_size": 1759552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25899608, "raw_average_key_size": 28, "raw_value_size": 108318264, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924986, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287248, "oldest_key_time": 0, "file_creation_time": 1732287248, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232411, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:13.718952 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232412.sst) /dev/shm/ToplingTemp/Topling-322921-34TJuY 2024/11/22-22:54:14.795915 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232412.sst): first pass time = 1.08's, 124.632'MB/sec 2024/11/22-22:54:16.984887 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232412.sst): old prealloc_size = 147639500, real_size = 32560528 2024/11/22-22:54:16.985557 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232412.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927906 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.65 avg-zval = 32.94 2024/11/22-22:54:16.986390 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232412.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:18.654858 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232412.sst): second pass time = 2.19's, 49.478'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.67's, 21.954'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2718.007'MB/sec (index lex order gen) rebuild zvType time = 1.41's, 0.165'MB/sec write SST data time = 0.26's, 139.529'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30861.079'MB/sec zip my value time = 2.19's, unzip length = 0.108'GB zip my value throughput = 49.478'MB/sec zip pipeline throughput = 95.744'MB/sec entries = 927906 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.65 avg-zval = 32.94 usrkeys = 927906 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.65 avg-zval = 32.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927906 bytes = 3247680 } seq expand size = 6495342 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5172 value = 3.5413 dict = 1.00 all = 3.6610 } Zip/UnZip{ index = 0.0951 value = 0.2824 dict = 1.00 all = 0.2732 } ---------------------------- total value len = 0.459118 GB avg = 0.124 KB (by entry num) total key len = 0.103671 GB avg = 0.028 KB total ukey len = 0.074051 GB avg = 0.020 KB total ukey num = 0.003702549 Billion total entry num = 0.003702549 Billion write speed all = 0.011140906 MB/sec (with seq num) write speed all = 0.010554545 MB/sec (without seq num) 2024/11/22-22:54:18.693305 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232412: 927906 keys, 36661740 bytes, temperature: kWarm 2024/11/22-22:54:18.693381 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287258693339, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232412, "file_size": 36661740, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3624093942, "table_properties": {"data_size": 30563840, "index_size": 1764544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25981368, "raw_average_key_size": 28, "raw_value_size": 108236488, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927906, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287253, "oldest_key_time": 0, "file_creation_time": 1732287253, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232412, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:18.747469 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232413.sst) /dev/shm/ToplingTemp/Topling-322921-fT5LHW 2024/11/22-22:54:19.665373 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232413.sst): first pass time = 0.92's, 146.228'MB/sec 2024/11/22-22:54:22.422739 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232413.sst): old prealloc_size = 147639500, real_size = 33368096 2024/11/22-22:54:22.423382 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232413.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927076 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.78 avg-zval = 33.84 2024/11/22-22:54:22.424197 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232413.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:23.552761 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232413.sst): second pass time = 2.76's, 39.273'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.13's, 33.156'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2959.737'MB/sec (index lex order gen) rebuild zvType time = 0.96's, 0.240'MB/sec write SST data time = 0.16's, 227.670'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 22665.841'MB/sec zip my value time = 2.76's, unzip length = 0.108'GB zip my value throughput = 39.273'MB/sec zip pipeline throughput = 94.222'MB/sec entries = 927076 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 33.84 usrkeys = 927076 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.78 avg-zval = 33.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927076 bytes = 3244784 } seq expand size = 6489532 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5113 value = 3.4508 dict = 1.00 all = 3.5823 } Zip/UnZip{ index = 0.0951 value = 0.2898 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 0.573867 GB avg = 0.124 KB (by entry num) total key len = 0.129630 GB avg = 0.028 KB total ukey len = 0.092592 GB avg = 0.020 KB total ukey num = 0.004629625 Billion total entry num = 0.004629625 Billion write speed all = 0.013924981 MB/sec (with seq num) write speed all = 0.013191872 MB/sec (without seq num) 2024/11/22-22:54:23.587548 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232413: 927076 keys, 37467150 bytes, temperature: kWarm 2024/11/22-22:54:23.587606 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287263587575, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232413, "file_size": 37467150, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3624093962, "table_properties": {"data_size": 31372176, "index_size": 1763968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958128, "raw_average_key_size": 28, "raw_value_size": 108259644, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927076, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287258, "oldest_key_time": 0, "file_creation_time": 1732287258, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232413, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:23.626882 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232414.sst) /dev/shm/ToplingTemp/Topling-322921-VxKSrZ 2024/11/22-22:54:24.751751 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232414.sst): first pass time = 1.12's, 119.324'MB/sec 2024/11/22-22:54:26.074812 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232414.sst): old prealloc_size = 147639500, real_size = 32684192 2024/11/22-22:54:26.075380 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232414.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924724 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.14 avg-zval = 33.19 2024/11/22-22:54:26.076130 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232414.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:26.967601 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232414.sst): second pass time = 1.32's, 81.916'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.193'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3226.660'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.303'MB/sec write SST data time = 0.13's, 283.643'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35111.183'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.916'MB/sec zip pipeline throughput = 81.916'MB/sec entries = 924724 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.14 avg-zval = 33.19 usrkeys = 924724 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.14 avg-zval = 33.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924724 bytes = 3236544 } seq expand size = 6473068 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5136 value = 3.5292 dict = 1.00 all = 3.6495 } Zip/UnZip{ index = 0.0951 value = 0.2833 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 0.688666 GB avg = 0.124 KB (by entry num) total key len = 0.155522 GB avg = 0.028 KB total ukey len = 0.111087 GB avg = 0.020 KB total ukey num = 0.005554349 Billion total entry num = 0.005554349 Billion write speed all = 0.016708683 MB/sec (with seq num) write speed all = 0.015829202 MB/sec (without seq num) 2024/11/22-22:54:27.002255 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232414: 924724 keys, 36776660 bytes, temperature: kWarm 2024/11/22-22:54:27.002317 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287267002285, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232414, "file_size": 36776660, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3465649241, "largest_seqno": 3624093980, "table_properties": {"data_size": 30693712, "index_size": 1759104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25892272, "raw_average_key_size": 28, "raw_value_size": 108325596, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924724, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287263, "oldest_key_time": 0, "file_creation_time": 1732287263, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232414, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:27.057519 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232415.sst) /dev/shm/ToplingTemp/Topling-322921-wJBoMX 2024/11/22-22:54:27.996546 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232415.sst): first pass time = 0.94's, 142.939'MB/sec 2024/11/22-22:54:29.450260 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232415.sst): old prealloc_size = 147639500, real_size = 33595008 2024/11/22-22:54:29.450821 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232415.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925104 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.08 avg-zval = 34.16 2024/11/22-22:54:29.451534 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232415.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:30.393447 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232415.sst): second pass time = 1.45's, 74.543'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.960'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3259.596'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.13's, 291.261'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36181.386'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.543'MB/sec zip pipeline throughput = 74.543'MB/sec entries = 925104 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 34.16 usrkeys = 925104 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 34.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925104 bytes = 3237872 } seq expand size = 6475728 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5133 value = 3.4273 dict = 1.00 all = 3.5612 } Zip/UnZip{ index = 0.0951 value = 0.2918 dict = 1.00 all = 0.2808 } ---------------------------- total value len = 0.803456 GB avg = 0.124 KB (by entry num) total key len = 0.181425 GB avg = 0.028 KB total ukey len = 0.129589 GB avg = 0.020 KB total ukey num = 0.006479453 Billion total entry num = 0.006479453 Billion write speed all = 0.019492055 MB/sec (with seq num) write speed all = 0.018466162 MB/sec (without seq num) 2024/11/22-22:54:30.429578 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232415: 925104 keys, 37688576 bytes, temperature: kWarm 2024/11/22-22:54:30.429643 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287270429610, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232415, "file_size": 37688576, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3624093995, "table_properties": {"data_size": 31603696, "index_size": 1759872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25902912, "raw_average_key_size": 28, "raw_value_size": 108314868, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925104, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287267, "oldest_key_time": 0, "file_creation_time": 1732287267, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232415, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:30.473310 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232416.sst) /dev/shm/ToplingTemp/Topling-322921-CkEl1V 2024/11/22-22:54:31.457871 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232416.sst): first pass time = 0.98's, 136.329'MB/sec 2024/11/22-22:54:32.812248 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232416.sst): old prealloc_size = 147639500, real_size = 33587200 2024/11/22-22:54:32.812811 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232416.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924774 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.14 avg-zval = 34.17 2024/11/22-22:54:32.813514 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232416.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:33.754098 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232416.sst): second pass time = 1.35's, 80.024'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.884'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3245.838'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.13's, 292.040'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35319.507'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.024'MB/sec zip pipeline throughput = 80.024'MB/sec entries = 924774 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.14 avg-zval = 34.17 usrkeys = 924774 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.14 avg-zval = 34.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924774 bytes = 3121120 } seq expand size = 6473418 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5130 value = 3.4284 dict = 1.00 all = 3.5730 } Zip/UnZip{ index = 0.0951 value = 0.2917 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 0.918254 GB avg = 0.124 KB (by entry num) total key len = 0.207318 GB avg = 0.028 KB total ukey len = 0.148085 GB avg = 0.020 KB total ukey num = 0.007404227 Billion total entry num = 0.007404227 Billion write speed all = 0.022275032 MB/sec (with seq num) write speed all = 0.021102796 MB/sec (without seq num) 2024/11/22-22:54:33.788753 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232416: 924774 keys, 37564408 bytes, temperature: kWarm 2024/11/22-22:54:33.788821 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287273788791, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232416, "file_size": 37564408, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3499102480, "largest_seqno": 3624094005, "table_properties": {"data_size": 31596528, "index_size": 1759296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25893672, "raw_average_key_size": 28, "raw_value_size": 108324056, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924774, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287270, "oldest_key_time": 0, "file_creation_time": 1732287270, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232416, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:33.824228 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232417.sst) /dev/shm/ToplingTemp/Topling-322921-vEBIQW 2024/11/22-22:54:34.761906 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232417.sst): first pass time = 0.94's, 143.144'MB/sec 2024/11/22-22:54:36.185839 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232417.sst): old prealloc_size = 147639500, real_size = 35946000 2024/11/22-22:54:36.186392 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232417.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927401 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0339 GB avg-val = 116.72 avg-zval = 36.59 2024/11/22-22:54:36.187152 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232417.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:37.052165 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232417.sst): second pass time = 1.42's, 76.059'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 46.128'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3331.057'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 299.522'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34943.524'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.059'MB/sec zip pipeline throughput = 76.059'MB/sec entries = 927401 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.72 avg-zval = 36.59 usrkeys = 927401 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.00 avg-zval = 36.59 TagRS{ kinds = RS_Key0_TagN bytes = 145000 } TagArray{ size = 830843 bytes = 3011824 } seq expand size = 5815901 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4422 value = 3.1897 dict = 1.00 all = 3.3587 } Zip/UnZip{ index = 0.0958 value = 0.3135 dict = 1.00 all = 0.2977 } ---------------------------- total value len = 1.032320 GB avg = 0.124 KB (by entry num) total key len = 0.233286 GB avg = 0.028 KB total ukey len = 0.166633 GB avg = 0.020 KB total ukey num = 0.008331628 Billion total entry num = 0.008331628 Billion write speed all = 0.025044658 MB/sec (with seq num) write speed all = 0.023725683 MB/sec (without seq num) 2024/11/22-22:54:37.089157 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232417: 927401 keys, 39960842 bytes, temperature: kWarm 2024/11/22-22:54:37.089220 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287277089187, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232417, "file_size": 39960842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3624094019, "table_properties": {"data_size": 33937728, "index_size": 1776256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25967228, "raw_average_key_size": 28, "raw_value_size": 108250526, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927401, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287273, "oldest_key_time": 0, "file_creation_time": 1732287273, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232417, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:37.125042 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232418.sst) /dev/shm/ToplingTemp/Topling-322921-uYKvr0 2024/11/22-22:54:38.128009 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232418.sst): first pass time = 1.00's, 133.826'MB/sec 2024/11/22-22:54:39.739497 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232418.sst): old prealloc_size = 147639500, real_size = 36442352 2024/11/22-22:54:39.740105 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232418.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926983 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.79 avg-zval = 37.15 2024/11/22-22:54:39.740162 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232418.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:40.408672 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232418.sst): second pass time = 1.61's, 67.212'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.67's, 55.742'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3001.344'MB/sec (index lex order gen) rebuild zvType time = 0.54's, 0.426'MB/sec write SST data time = 0.12's, 299.153'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31878.124'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.212'MB/sec zip pipeline throughput = 67.212'MB/sec entries = 926983 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.79 avg-zval = 37.15 usrkeys = 926983 avg-key = 20.00 avg-zkey = 1.91 avg-val = 116.79 avg-zval = 37.15 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4458 value = 3.1439 dict = 1.00 all = 3.5983 } Zip/UnZip{ index = 0.0957 value = 0.3181 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 1.140582 GB avg = 0.123 KB (by entry num) total key len = 0.259241 GB avg = 0.028 KB total ukey len = 0.185172 GB avg = 0.020 KB total ukey num = 0.009258611 Billion total entry num = 0.009258611 Billion write speed all = 0.027698812 MB/sec (with seq num) write speed all = 0.026233184 MB/sec (without seq num) 2024/11/22-22:54:40.442196 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232418: 926983 keys, 37300288 bytes, temperature: kWarm 2024/11/22-22:54:40.442259 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287280442227, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232418, "file_size": 37300288, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34435552, "index_size": 1774848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955524, "raw_average_key_size": 28, "raw_value_size": 108262326, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926983, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287277, "oldest_key_time": 0, "file_creation_time": 1732287277, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232418, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:40.477730 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232419.sst) /dev/shm/ToplingTemp/Topling-322921-ho02hZ 2024/11/22-22:54:41.274456 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232419.sst): first pass time = 0.80's, 168.470'MB/sec 2024/11/22-22:54:42.839119 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232419.sst): old prealloc_size = 147639500, real_size = 36406336 2024/11/22-22:54:42.839666 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232419.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926991 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.79 avg-zval = 37.12 2024/11/22-22:54:42.839708 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232419.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:43.484750 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232419.sst): second pass time = 1.56's, 69.222'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 57.719'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3345.754'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.446'MB/sec write SST data time = 0.13's, 295.767'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33843.566'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.222'MB/sec zip pipeline throughput = 69.222'MB/sec entries = 926991 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 37.12 usrkeys = 926991 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 37.12 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5130 value = 3.1462 dict = 1.00 all = 3.6018 } Zip/UnZip{ index = 0.0951 value = 0.3178 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 1.248844 GB avg = 0.123 KB (by entry num) total key len = 0.285197 GB avg = 0.028 KB total ukey len = 0.203712 GB avg = 0.020 KB total ukey num = 0.010185602 Billion total entry num = 0.010185602 Billion write speed all = 0.030352781 MB/sec (with seq num) write speed all = 0.028740510 MB/sec (without seq num) 2024/11/22-22:54:43.518082 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232419: 926991 keys, 37264488 bytes, temperature: kWarm 2024/11/22-22:54:43.518144 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287283518111, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232419, "file_size": 37264488, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34410864, "index_size": 1763520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955748, "raw_average_key_size": 28, "raw_value_size": 108262082, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926991, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287280, "oldest_key_time": 0, "file_creation_time": 1732287280, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232419, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:43.556337 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232420.sst) /dev/shm/ToplingTemp/Topling-322921-tkhFDY 2024/11/22-22:54:44.472556 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232420.sst): first pass time = 0.92's, 146.497'MB/sec 2024/11/22-22:54:45.820913 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232420.sst): old prealloc_size = 147639500, real_size = 33727376 2024/11/22-22:54:45.821438 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232420.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921172 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.70 avg-zval = 34.46 2024/11/22-22:54:45.822096 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232420.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:46.686344 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232420.sst): second pass time = 1.35's, 80.499'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.042'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3494.017'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.314'MB/sec write SST data time = 0.13's, 285.599'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33991.843'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.499'MB/sec zip pipeline throughput = 80.499'MB/sec entries = 921172 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.70 avg-zval = 34.46 usrkeys = 921172 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.98 avg-zval = 34.46 TagRS{ kinds = RS_Key0_TagN bytes = 144040 } TagArray{ size = 693933 bytes = 2515520 } seq expand size = 4857531 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5103 value = 3.4156 dict = 1.00 all = 3.6032 } Zip/UnZip{ index = 0.0951 value = 0.2928 dict = 1.00 all = 0.2775 } ---------------------------- total value len = 1.362127 GB avg = 0.123 KB (by entry num) total key len = 0.310990 GB avg = 0.028 KB total ukey len = 0.222135 GB avg = 0.020 KB total ukey num = 0.011106774 Billion total entry num = 0.011106774 Billion write speed all = 0.033102451 MB/sec (with seq num) write speed all = 0.031344479 MB/sec (without seq num) 2024/11/22-22:54:46.721543 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232420: 921172 keys, 37249394 bytes, temperature: kWarm 2024/11/22-22:54:46.721607 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287286721576, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232420, "file_size": 37249394, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3608132111, "table_properties": {"data_size": 31744000, "index_size": 1752896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25792816, "raw_average_key_size": 28, "raw_value_size": 108425008, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921172, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287283, "oldest_key_time": 0, "file_creation_time": 1732287283, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232420, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:46.758644 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232421.sst) /dev/shm/ToplingTemp/Topling-322921-tI1FxZ 2024/11/22-22:54:47.612499 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232421.sst): first pass time = 0.85's, 157.199'MB/sec 2024/11/22-22:54:48.947418 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232421.sst): old prealloc_size = 147639500, real_size = 32637936 2024/11/22-22:54:48.947964 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232421.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931067 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.15 avg-zval = 32.90 2024/11/22-22:54:48.948710 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232421.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:49.893738 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232421.sst): second pass time = 1.33's, 81.055'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.710'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3455.390'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.13's, 287.037'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33710.159'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.055'MB/sec zip pipeline throughput = 81.055'MB/sec entries = 931067 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.15 avg-zval = 32.90 usrkeys = 931067 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 32.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931067 bytes = 3142368 } seq expand size = 6517469 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5130 value = 3.5304 dict = 1.00 all = 3.6640 } Zip/UnZip{ index = 0.0951 value = 0.2833 dict = 1.00 all = 0.2729 } ---------------------------- total value len = 1.476792 GB avg = 0.123 KB (by entry num) total key len = 0.337060 GB avg = 0.028 KB total ukey len = 0.240757 GB avg = 0.020 KB total ukey num = 0.012037841 Billion total entry num = 0.012037841 Billion write speed all = 0.035884607 MB/sec (with seq num) write speed all = 0.033979388 MB/sec (without seq num) 2024/11/22-22:54:49.938320 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232421: 931067 keys, 36631748 bytes, temperature: kWarm 2024/11/22-22:54:49.938384 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287289938351, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232421, "file_size": 36631748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3509327862, "largest_seqno": 3624093721, "table_properties": {"data_size": 30633696, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26069876, "raw_average_key_size": 28, "raw_value_size": 108147942, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931067, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287286, "oldest_key_time": 0, "file_creation_time": 1732287286, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232421, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:49.974389 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232422.sst) /dev/shm/ToplingTemp/Topling-322921-PDixSW 2024/11/22-22:54:50.893736 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232422.sst): first pass time = 0.92's, 145.999'MB/sec 2024/11/22-22:54:52.261660 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232422.sst): old prealloc_size = 147639500, real_size = 33722128 2024/11/22-22:54:52.262245 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232422.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931124 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0317 GB avg-val = 116.15 avg-zval = 34.05 2024/11/22-22:54:52.263050 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232422.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:53.190067 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232422.sst): second pass time = 1.37's, 79.099'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.876'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3179.429'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.13's, 297.338'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 22683.298'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.099'MB/sec zip pipeline throughput = 79.099'MB/sec entries = 931124 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 34.05 usrkeys = 931124 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 34.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931124 bytes = 3375344 } seq expand size = 6517868 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4446 value = 3.4109 dict = 1.00 all = 3.5368 } Zip/UnZip{ index = 0.0957 value = 0.2932 dict = 1.00 all = 0.2827 } ---------------------------- total value len = 1.591457 GB avg = 0.123 KB (by entry num) total key len = 0.363131 GB avg = 0.028 KB total ukey len = 0.259379 GB avg = 0.020 KB total ukey num = 0.012968965 Billion total entry num = 0.012968965 Billion write speed all = 0.038666352 MB/sec (with seq num) write speed all = 0.036613898 MB/sec (without seq num) 2024/11/22-22:54:53.225979 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232422: 931124 keys, 37948780 bytes, temperature: kWarm 2024/11/22-22:54:53.226042 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287293226010, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232422, "file_size": 37948780, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3624093678, "table_properties": {"data_size": 31706176, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26071472, "raw_average_key_size": 28, "raw_value_size": 108146400, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931124, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287289, "oldest_key_time": 0, "file_creation_time": 1732287289, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232422, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:53.263250 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232423.sst) /dev/shm/ToplingTemp/Topling-322921-By6iDY 2024/11/22-22:54:54.181959 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232423.sst): first pass time = 0.92's, 146.100'MB/sec 2024/11/22-22:54:55.543373 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232423.sst): old prealloc_size = 147639500, real_size = 34282528 2024/11/22-22:54:55.543958 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232423.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928476 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.56 avg-zval = 34.77 2024/11/22-22:54:55.544718 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232423.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:54:56.444621 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232423.sst): second pass time = 1.36's, 79.530'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.592'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3137.026'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.13's, 302.030'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33607.372'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.530'MB/sec zip pipeline throughput = 79.530'MB/sec entries = 928476 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 34.77 usrkeys = 928476 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 34.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928476 bytes = 3249680 } seq expand size = 6499332 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5145 value = 3.3521 dict = 1.00 all = 3.4966 } Zip/UnZip{ index = 0.0951 value = 0.2983 dict = 1.00 all = 0.2860 } ---------------------------- total value len = 1.706177 GB avg = 0.123 KB (by entry num) total key len = 0.389128 GB avg = 0.028 KB total ukey len = 0.277949 GB avg = 0.020 KB total ukey num = 0.013897441 Billion total entry num = 0.013897441 Billion write speed all = 0.041447400 MB/sec (with seq num) write speed all = 0.039248148 MB/sec (without seq num) 2024/11/22-22:54:56.478159 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232423: 928476 keys, 38385472 bytes, temperature: kWarm 2024/11/22-22:54:56.478223 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287296478189, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232423, "file_size": 38385472, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3624093089, "table_properties": {"data_size": 32284176, "index_size": 1766080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997328, "raw_average_key_size": 28, "raw_value_size": 108220468, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928476, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287293, "oldest_key_time": 0, "file_creation_time": 1732287293, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232423, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:56.513542 140024844146240 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232424.sst) /dev/shm/ToplingTemp/Topling-322921-k0X5fX 2024/11/22-22:54:56.587993 140024844146240 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232424.sst): first pass time = 0.07's, 236.265'MB/sec 2024/11/22-22:54:56.742113 140024844146240 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232424.sst): old prealloc_size = 147639500, real_size = 5253856 2024/11/22-22:54:56.742233 140024844146240 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232424.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 120413 prefix = 4 raw-key = 0.0019 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0142 GB zip-val = 0.0050 GB avg-val = 118.00 avg-zval = 41.45 2024/11/22-22:54:56.742368 140024844146240 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232424.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/22-22:54:56.863108 140024844146240 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00/232424.sst): second pass time = 0.15's, 92.529'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 47.527'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2444.241'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.291'MB/sec write SST data time = 0.02's, 326.884'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.148'MB, throughput = 5156.633'MB/sec zip my value time = 0.15's, unzip length = 0.014'GB zip my value throughput = 92.529'MB/sec zip pipeline throughput = 92.529'MB/sec entries = 120413 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 41.45 usrkeys = 120413 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 41.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 120413 bytes = 376304 } seq expand size = 842891 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0024 GB value = 0.0142 GB dict = 0.15 MB all = 0.0176 GB } __ZipSize{ index = 0.0002 GB value = 0.0050 GB dict = 0.15 MB all = 0.0057 GB } UnZip/Zip{ index = 10.3804 value = 2.8465 dict = 1.00 all = 3.0575 } Zip/UnZip{ index = 0.0963 value = 0.3513 dict = 1.00 all = 0.3271 } ---------------------------- total value len = 1.721228 GB avg = 0.123 KB (by entry num) total key len = 0.392500 GB avg = 0.028 KB total ukey len = 0.280357 GB avg = 0.020 KB total ukey num = 0.014017854 Billion total entry num = 0.014017854 Billion write speed all = 0.041811484 MB/sec (with seq num) write speed all = 0.039593196 MB/sec (without seq num) 2024/11/22-22:54:56.873848 140024844146240 [compaction_job.cc:1948] [default] [JOB 13283] Generated table #232424: 120413 keys, 5749883 bytes, temperature: kWarm 2024/11/22-22:54:56.873906 140024844146240 EVENT_LOG_v1 {"time_micros": 1732287296873876, "cf_name": "default", "job": 13283, "event": "table_file_creation", "file_number": 232424, "file_size": 5749883, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3589656452, "largest_seqno": 3614401610, "table_properties": {"data_size": 4991568, "index_size": 232000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3371564, "raw_average_key_size": 28, "raw_value_size": 14208734, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 120413, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732287296, "oldest_key_time": 0, "file_creation_time": 1732287296, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232424, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:54:56.890525 140024844146240 [compaction_job.cc:703] job-13283: subcompact[0], size: 0.568437 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13283/att-00 [232409,232410,232411,232412,232413,232414,232415,232416,232417,232418,232419,232420,232421,232422,232423,232424] 2024/11/22-22:54:56.989882 140024844146240 (Original Log Time 2024/11/22-22:53:59.945899) EVENT_LOG_v1 {"time_micros": 1732287239945839, "job": 13283, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [232405], "files_L3": [232334, 232335, 232336, 232337, 232338, 232339, 232340, 232341, 232342, 232343, 232344, 232345, 232346, 232347, 232360], "score": 0.99886, "input_data_size": 591415258, "oldest_snapshot_seqno": 3624136355} 2024/11/22-22:54:56.989884 140024844146240 (Original Log Time 2024/11/22-22:54:56.989079) [compaction_job.cc:2022] [default] [JOB 13283] Compacted 1@2 + 15@3 files to L3 => 568436981 bytes 2024/11/22-22:54:56.989886 140024844146240 (Original Log Time 2024/11/22-22:54:56.989817) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.4 rd, 10.0 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.1, 536.9 +0.0 blob) out(542.1 +0.0 blob), read-write-amplify(40.8) write-amplify(20.0) OK, records in: 14129963, records dropped: 112109 output_compression: Snappy 2024/11/22-22:54:56.989891 140024844146240 (Original Log Time 2024/11/22-22:54:56.989862) EVENT_LOG_v1 {"time_micros": 1732287296989841, "cf": "default", "job": 13283, "event": "compaction_finished", "compaction_time_micros": 56944632, "compaction_time_cpu_micros": 24880498, "output_level": 3, "num_output_files": 16, "total_output_size": 568436981, "num_input_records": 14129963, "num_output_records": 14017854, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 411047422, "file_range_sync_nanos": 0, "file_fsync_nanos": 16410, "file_prepare_write_nanos": 35743, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/22-22:54:57.115557 140024844146240 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732287239,"executesMs":57218,"compactionJobId":13283,"attempt":0,"compactionInputRawBytes":2046766938,"compactionInputZipBytes":591415258,"compactionOutputRawBytes":0,"compactionOutputZipBytes":568436981,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-791ffe396357df9439799d7b4bc4cb73-ce5f46da73e8623a-00"}}, response = 2024/11/22-22:54:57.115645 140024844146240 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13283/att-00: olev 3, work 57.219 s, result 4.924 ms, install 0.834 ms, input{raw 1.906 GiB zip 564.018 MiB}