2024/11/23-03:54:54.934988 140024852538944 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00: fork to child time = 0.008814 sec 2024/11/23-03:54:54.935859 140024852538944 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00 2024/11/23-03:54:54.942457 140024852538944 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00 2024/11/23-03:54:54.943580 140024852538944 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-03:54:54.943828 140024852538944 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-03:54:54.994003 140024852538944 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-18771/att-00/MANIFEST-206347 2024/11/23-03:54:55.023817 140024852538944 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-18771/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 243605, last_sequence is 3771369001, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 243412 2024/11/23-03:54:55.023823 140024852538944 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-03:54:55.024073 140024852538944 [version_set.cc:5721] Creating manifest 243605 2024/11/23-03:54:55.055350 140024852538944 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-03:54:55.076799 140024852538944 [compaction_job.cc:2437] [default] [JOB 18771] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-03:54:55.076813 140024852538944 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 14027 Base level 2, inputs: [243572(27MB)], [243576(9963KB) 243477(35MB) 243478(36MB) 243479(33MB) 243480(33MB) 243481(35MB) 243482(33MB) 243483(34MB) 243484(34MB) 243485(35MB) 243486(35MB) 243503(36MB) 243504(35MB) 243505(35MB) 243599(37MB)] 2024/11/23-03:54:55.082676 140024852538944 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-03:54:55.082704 140024852538944 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-03:54:55.082722 140024852538944 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-03:54:55.082727 140024852538944 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-03:54:55.082737 140024852538944 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-03:54:55.392280 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243606.sst) /dev/shm/ToplingTemp/Topling-401183-MQbgvN 2024/11/23-03:54:56.433253 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243606.sst): first pass time = 1.04's, 128.942'MB/sec 2024/11/23-03:54:58.014566 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243606.sst): old prealloc_size = 147639500, real_size = 35023984 2024/11/23-03:54:58.015292 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243606.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929792 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0330 GB avg-val = 116.35 avg-zval = 35.50 2024/11/23-03:54:58.015962 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243606.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:54:58.986248 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243606.sst): second pass time = 1.58's, 68.455'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 39.372'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2525.121'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.16's, 238.822'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36341.710'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.455'MB/sec zip pipeline throughput = 68.455'MB/sec entries = 929792 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.35 avg-zval = 35.50 usrkeys = 929792 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.15 avg-zval = 35.50 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 637803 bytes = 2232320 } seq expand size = 4464621 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4450 value = 3.2772 dict = 1.00 all = 3.5083 } Zip/UnZip{ index = 0.0957 value = 0.3051 dict = 1.00 all = 0.2850 } ---------------------------- total value len = 0.112648 GB avg = 0.121 KB (by entry num) total key len = 0.026034 GB avg = 0.028 KB total ukey len = 0.018596 GB avg = 0.020 KB total ukey num = 0.000929792 Billion total entry num = 0.000929792 Billion write speed all = 0.002022910 MB/sec (with seq num) write speed all = 0.001914409 MB/sec (without seq num) 2024/11/23-03:54:59.024010 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243606: 929792 keys, 38257480 bytes, temperature: kWarm 2024/11/23-03:54:59.024127 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305299024054, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243606, "file_size": 38257480, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3771167952, "table_properties": {"data_size": 33010976, "index_size": 1780352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26034176, "raw_average_key_size": 28, "raw_value_size": 108183580, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929792, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305295, "oldest_key_time": 0, "file_creation_time": 1732305295, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243606, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:54:59.080489 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243607.sst) /dev/shm/ToplingTemp/Topling-401183-DMeEFN 2024/11/23-03:55:00.260320 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243607.sst): first pass time = 1.18's, 113.765'MB/sec 2024/11/23-03:55:02.166788 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243607.sst): old prealloc_size = 147639500, real_size = 34960720 2024/11/23-03:55:02.167451 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243607.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081415 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.11 avg-zval = 30.16 2024/11/23-03:55:02.167491 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243607.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:02.931074 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243607.sst): second pass time = 1.90's, 54.647'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 46.757'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3209.282'MB/sec (index lex order gen) rebuild zvType time = 0.50's, 0.538'MB/sec write SST data time = 0.26's, 136.890'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.033'MB, throughput = 29821.446'MB/sec zip my value time = 1.90's, unzip length = 0.104'GB zip my value throughput = 54.647'MB/sec zip pipeline throughput = 123.852'MB/sec entries = 1081415 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.11 avg-zval = 30.16 usrkeys = 1081415 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.11 avg-zval = 30.16 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.03 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4484 value = 3.1863 dict = 1.00 all = 3.7560 } Zip/UnZip{ index = 0.0957 value = 0.3138 dict = 1.00 all = 0.2662 } ---------------------------- total value len = 0.216586 GB avg = 0.108 KB (by entry num) total key len = 0.056314 GB avg = 0.028 KB total ukey len = 0.040224 GB avg = 0.020 KB total ukey num = 0.002011207 Billion total entry num = 0.002011207 Billion write speed all = 0.003980466 MB/sec (with seq num) write speed all = 0.003745786 MB/sec (without seq num) 2024/11/23-03:55:02.974237 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243607: 1081415 keys, 35734414 bytes, temperature: kWarm 2024/11/23-03:55:02.974341 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305302974295, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243607, "file_size": 35734414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32620160, "index_size": 2070016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30279620, "raw_average_key_size": 28, "raw_value_size": 103938115, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081415, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305299, "oldest_key_time": 0, "file_creation_time": 1732305299, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243607, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:03.012924 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243608.sst) /dev/shm/ToplingTemp/Topling-401183-SYMrjK 2024/11/23-03:55:03.848464 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243608.sst): first pass time = 0.84's, 160.644'MB/sec 2024/11/23-03:55:05.209238 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243608.sst): old prealloc_size = 147639500, real_size = 31546944 2024/11/23-03:55:05.209772 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243608.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921543 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0296 GB avg-val = 117.64 avg-zval = 32.07 2024/11/23-03:55:05.210576 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243608.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:06.113188 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243608.sst): second pass time = 1.36's, 79.752'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 39.650'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3365.197'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.18's, 202.703'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 29589.570'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.752'MB/sec zip pipeline throughput = 79.752'MB/sec entries = 921543 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.64 avg-zval = 32.07 usrkeys = 921543 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.53 avg-zval = 32.07 TagRS{ kinds = RS_Key0_TagN bytes = 144080 } TagArray{ size = 906544 bytes = 3286240 } seq expand size = 6345808 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.4436 value = 3.6687 dict = 1.00 all = 3.7449 } Zip/UnZip{ index = 0.0958 value = 0.2726 dict = 1.00 all = 0.2670 } ---------------------------- total value len = 0.331347 GB avg = 0.113 KB (by entry num) total key len = 0.082117 GB avg = 0.028 KB total ukey len = 0.058655 GB avg = 0.020 KB total ukey num = 0.002932750 Billion total entry num = 0.002932750 Billion write speed all = 0.006030419 MB/sec (with seq num) write speed all = 0.005688223 MB/sec (without seq num) 2024/11/23-03:55:06.146183 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243608: 921543 keys, 35840150 bytes, temperature: kWarm 2024/11/23-03:55:06.146264 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305306146220, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243608, "file_size": 35840150, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3771167654, "table_properties": {"data_size": 29551600, "index_size": 1764800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25803204, "raw_average_key_size": 28, "raw_value_size": 108414590, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921543, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305302, "oldest_key_time": 0, "file_creation_time": 1732305302, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243608, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:06.195067 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243609.sst) /dev/shm/ToplingTemp/Topling-401183-rp29OL 2024/11/23-03:55:07.100268 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243609.sst): first pass time = 0.91's, 148.284'MB/sec 2024/11/23-03:55:08.462426 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243609.sst): old prealloc_size = 147639500, real_size = 32887488 2024/11/23-03:55:08.462971 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243609.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925439 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.03 avg-zval = 33.38 2024/11/23-03:55:08.463756 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243609.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:09.504953 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243609.sst): second pass time = 1.36's, 79.579'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 35.584'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3380.313'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.263'MB/sec write SST data time = 0.16's, 228.656'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32540.832'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.579'MB/sec zip pipeline throughput = 79.579'MB/sec entries = 925439 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.03 avg-zval = 33.38 usrkeys = 925439 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.03 avg-zval = 33.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925439 bytes = 3354736 } seq expand size = 6478073 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5114 value = 3.5056 dict = 1.00 all = 3.6180 } Zip/UnZip{ index = 0.0951 value = 0.2853 dict = 1.00 all = 0.2764 } ---------------------------- total value len = 0.446130 GB avg = 0.116 KB (by entry num) total key len = 0.108029 GB avg = 0.028 KB total ukey len = 0.077164 GB avg = 0.020 KB total ukey num = 0.003858189 Billion total entry num = 0.003858189 Billion write speed all = 0.008082086 MB/sec (with seq num) write speed all = 0.007631931 MB/sec (without seq num) 2024/11/23-03:55:09.542920 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243609: 925439 keys, 37096914 bytes, temperature: kWarm 2024/11/23-03:55:09.542985 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305309542952, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243609, "file_size": 37096914, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3771165971, "table_properties": {"data_size": 30895088, "index_size": 1760832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912292, "raw_average_key_size": 28, "raw_value_size": 108305554, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925439, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305306, "oldest_key_time": 0, "file_creation_time": 1732305306, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243609, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:09.586211 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243610.sst) /dev/shm/ToplingTemp/Topling-401183-TzvzNM 2024/11/23-03:55:10.484045 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243610.sst): first pass time = 0.90's, 149.497'MB/sec 2024/11/23-03:55:11.875038 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243610.sst): old prealloc_size = 147639500, real_size = 32905360 2024/11/23-03:55:11.875577 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243610.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930926 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.18 avg-zval = 33.19 2024/11/23-03:55:11.876409 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243610.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:12.927622 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243610.sst): second pass time = 1.39's, 77.798'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 35.277'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3447.575'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.253'MB/sec write SST data time = 0.13's, 284.205'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31347.869'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.798'MB/sec zip pipeline throughput = 77.798'MB/sec entries = 930926 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.18 avg-zval = 33.19 usrkeys = 930926 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.18 avg-zval = 33.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930926 bytes = 3374624 } seq expand size = 6516482 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5114 value = 3.4999 dict = 1.00 all = 3.6147 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 0.560799 GB avg = 0.117 KB (by entry num) total key len = 0.134095 GB avg = 0.028 KB total ukey len = 0.095782 GB avg = 0.020 KB total ukey num = 0.004789115 Billion total entry num = 0.004789115 Billion write speed all = 0.010134106 MB/sec (with seq num) write speed all = 0.009575363 MB/sec (without seq num) 2024/11/23-03:55:12.964011 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243610: 930926 keys, 37131376 bytes, temperature: kWarm 2024/11/23-03:55:12.964079 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305312964043, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243610, "file_size": 37131376, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375334738, "largest_seqno": 3771168351, "table_properties": {"data_size": 30901184, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26065928, "raw_average_key_size": 28, "raw_value_size": 108151888, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930926, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305309, "oldest_key_time": 0, "file_creation_time": 1732305309, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243610, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:13.005774 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243611.sst) /dev/shm/ToplingTemp/Topling-401183-BMK7AM 2024/11/23-03:55:13.836475 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243611.sst): first pass time = 0.83's, 161.580'MB/sec 2024/11/23-03:55:15.210975 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243611.sst): old prealloc_size = 147639500, real_size = 32875104 2024/11/23-03:55:15.211530 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243611.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925229 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.06 avg-zval = 33.37 2024/11/23-03:55:15.212358 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243611.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:16.247602 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243611.sst): second pass time = 1.37's, 78.870'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 35.774'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3307.902'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.13's, 284.126'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32976.926'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.870'MB/sec zip pipeline throughput = 78.870'MB/sec entries = 925229 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.06 avg-zval = 33.37 usrkeys = 925229 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.06 avg-zval = 33.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925229 bytes = 3353968 } seq expand size = 6476603 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4433 value = 3.5084 dict = 1.00 all = 3.6193 } Zip/UnZip{ index = 0.0958 value = 0.2850 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 0.675587 GB avg = 0.118 KB (by entry num) total key len = 0.160002 GB avg = 0.028 KB total ukey len = 0.114287 GB avg = 0.020 KB total ukey num = 0.005714344 Billion total entry num = 0.005714344 Billion write speed all = 0.012185357 MB/sec (with seq num) write speed all = 0.011518700 MB/sec (without seq num) 2024/11/23-03:55:16.280583 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243611: 925229 keys, 37083928 bytes, temperature: kWarm 2024/11/23-03:55:16.280645 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305316280613, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243611, "file_size": 37083928, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3771168570, "table_properties": {"data_size": 30871696, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25906412, "raw_average_key_size": 28, "raw_value_size": 108311370, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925229, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305312, "oldest_key_time": 0, "file_creation_time": 1732305312, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243611, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:16.314904 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243612.sst) /dev/shm/ToplingTemp/Topling-401183-gXssiL 2024/11/23-03:55:17.124159 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243612.sst): first pass time = 0.81's, 165.860'MB/sec 2024/11/23-03:55:18.467147 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243612.sst): old prealloc_size = 147639500, real_size = 32324496 2024/11/23-03:55:18.467713 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243612.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932062 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 116.00 avg-zval = 32.53 2024/11/23-03:55:18.468498 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243612.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:19.493680 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243612.sst): second pass time = 1.34's, 80.550'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 35.383'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3266.017'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.260'MB/sec write SST data time = 0.13's, 280.346'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33764.912'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.550'MB/sec zip pipeline throughput = 80.550'MB/sec entries = 932062 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.00 avg-zval = 32.53 usrkeys = 932062 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.00 avg-zval = 32.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932062 bytes = 3145728 } seq expand size = 6524434 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5125 value = 3.5662 dict = 1.00 all = 3.6953 } Zip/UnZip{ index = 0.0951 value = 0.2804 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 0.790231 GB avg = 0.119 KB (by entry num) total key len = 0.186099 GB avg = 0.028 KB total ukey len = 0.132928 GB avg = 0.020 KB total ukey num = 0.006646406 Billion total entry num = 0.006646406 Billion write speed all = 0.014237122 MB/sec (with seq num) write speed all = 0.013461764 MB/sec (without seq num) 2024/11/23-03:55:19.526933 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243612: 932062 keys, 36320864 bytes, temperature: kWarm 2024/11/23-03:55:19.526995 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305319526964, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243612, "file_size": 36320864, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3771168888, "table_properties": {"data_size": 30318080, "index_size": 1773248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26097736, "raw_average_key_size": 28, "raw_value_size": 108120072, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 932062, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305316, "oldest_key_time": 0, "file_creation_time": 1732305316, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243612, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:19.561822 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243613.sst) /dev/shm/ToplingTemp/Topling-401183-gJXjtL 2024/11/23-03:55:20.383633 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243613.sst): first pass time = 0.82's, 163.325'MB/sec 2024/11/23-03:55:21.701398 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243613.sst): old prealloc_size = 147639500, real_size = 32282656 2024/11/23-03:55:21.701987 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243613.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923790 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0303 GB avg-val = 117.29 avg-zval = 32.79 2024/11/23-03:55:21.702767 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243613.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:22.700885 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243613.sst): second pass time = 1.32's, 82.286'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 36.508'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3102.962'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.265'MB/sec write SST data time = 0.13's, 284.209'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 30950.959'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.286'MB/sec zip pipeline throughput = 82.286'MB/sec entries = 923790 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.29 avg-zval = 32.79 usrkeys = 923790 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.29 avg-zval = 32.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923790 bytes = 3348752 } seq expand size = 6466530 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5141 value = 3.5766 dict = 1.00 all = 3.6783 } Zip/UnZip{ index = 0.0951 value = 0.2796 dict = 1.00 all = 0.2719 } ---------------------------- total value len = 0.905049 GB avg = 0.120 KB (by entry num) total key len = 0.211965 GB avg = 0.028 KB total ukey len = 0.151404 GB avg = 0.020 KB total ukey num = 0.007570196 Billion total entry num = 0.007570196 Billion write speed all = 0.016287858 MB/sec (with seq num) write speed all = 0.015404773 MB/sec (without seq num) 2024/11/23-03:55:22.735084 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243613: 923790 keys, 36488674 bytes, temperature: kWarm 2024/11/23-03:55:22.735170 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305322735116, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243613, "file_size": 36488674, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3771167808, "table_properties": {"data_size": 30294288, "index_size": 1757248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25866120, "raw_average_key_size": 28, "raw_value_size": 108351636, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923790, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305319, "oldest_key_time": 0, "file_creation_time": 1732305319, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243613, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:22.772746 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243614.sst) /dev/shm/ToplingTemp/Topling-401183-MhdXQM 2024/11/23-03:55:23.570987 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243614.sst): first pass time = 0.80's, 168.149'MB/sec 2024/11/23-03:55:24.929488 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243614.sst): old prealloc_size = 147639500, real_size = 32650192 2024/11/23-03:55:24.930031 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243614.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934014 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.70 avg-zval = 32.79 2024/11/23-03:55:24.931635 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243614.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:25.859862 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243614.sst): second pass time = 1.36's, 79.614'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.648'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3414.387'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.13's, 285.094'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34105.739'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.614'MB/sec zip pipeline throughput = 79.614'MB/sec entries = 934014 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.70 avg-zval = 32.79 usrkeys = 934014 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.70 avg-zval = 32.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934014 bytes = 3385808 } seq expand size = 6538098 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4433 value = 3.5283 dict = 1.00 all = 3.6387 } Zip/UnZip{ index = 0.0958 value = 0.2834 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 1.019653 GB avg = 0.120 KB (by entry num) total key len = 0.238118 GB avg = 0.028 KB total ukey len = 0.170084 GB avg = 0.020 KB total ukey num = 0.008504210 Billion total entry num = 0.008504210 Billion write speed all = 0.018339458 MB/sec (with seq num) write speed all = 0.017347464 MB/sec (without seq num) 2024/11/23-03:55:25.893270 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243614: 934014 keys, 36886410 bytes, temperature: kWarm 2024/11/23-03:55:25.893334 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305325893301, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243614, "file_size": 36886410, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3339428666, "largest_seqno": 3771168802, "table_properties": {"data_size": 30627776, "index_size": 1788736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26152392, "raw_average_key_size": 28, "raw_value_size": 108065352, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934014, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305322, "oldest_key_time": 0, "file_creation_time": 1732305322, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243614, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:25.929734 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243615.sst) /dev/shm/ToplingTemp/Topling-401183-uwj62M 2024/11/23-03:55:26.733928 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243615.sst): first pass time = 0.80's, 166.904'MB/sec 2024/11/23-03:55:28.078363 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243615.sst): old prealloc_size = 147639500, real_size = 33161360 2024/11/23-03:55:28.078931 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243615.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928014 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.63 avg-zval = 33.58 2024/11/23-03:55:28.082004 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243615.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:29.029680 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243615.sst): second pass time = 1.34's, 80.543'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.171'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3236.474'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 287.979'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32272.882'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.543'MB/sec zip pipeline throughput = 80.543'MB/sec entries = 928014 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.63 avg-zval = 33.58 usrkeys = 928014 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.63 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928014 bytes = 3248064 } seq expand size = 6496098 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5124 value = 3.4731 dict = 1.00 all = 3.6019 } Zip/UnZip{ index = 0.0951 value = 0.2879 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 1.134382 GB avg = 0.120 KB (by entry num) total key len = 0.264102 GB avg = 0.028 KB total ukey len = 0.188644 GB avg = 0.020 KB total ukey num = 0.009432224 Billion total entry num = 0.009432224 Billion write speed all = 0.020390253 MB/sec (with seq num) write speed all = 0.019290060 MB/sec (without seq num) 2024/11/23-03:55:29.063992 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243615: 928014 keys, 37263260 bytes, temperature: kWarm 2024/11/23-03:55:29.064056 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305329064024, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243615, "file_size": 37263260, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3771167921, "table_properties": {"data_size": 31163584, "index_size": 1765568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25984392, "raw_average_key_size": 28, "raw_value_size": 108233372, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928014, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305325, "oldest_key_time": 0, "file_creation_time": 1732305325, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243615, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:29.105628 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243616.sst) /dev/shm/ToplingTemp/Topling-401183-z1ahMN 2024/11/23-03:55:29.950620 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243616.sst): first pass time = 0.84's, 158.846'MB/sec 2024/11/23-03:55:31.273353 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243616.sst): old prealloc_size = 147639500, real_size = 32268400 2024/11/23-03:55:31.273936 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243616.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932500 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 115.93 avg-zval = 32.45 2024/11/23-03:55:31.276546 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243616.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:32.213346 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243616.sst): second pass time = 1.32's, 81.802'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 38.830'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3163.186'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.289'MB/sec write SST data time = 0.13's, 276.985'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28979.512'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.802'MB/sec zip pipeline throughput = 81.802'MB/sec entries = 932500 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.93 avg-zval = 32.45 usrkeys = 932500 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.93 avg-zval = 32.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932500 bytes = 3380320 } seq expand size = 6527500 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5129 value = 3.5725 dict = 1.00 all = 3.6773 } Zip/UnZip{ index = 0.0951 value = 0.2799 dict = 1.00 all = 0.2719 } ---------------------------- total value len = 1.249018 GB avg = 0.121 KB (by entry num) total key len = 0.290212 GB avg = 0.028 KB total ukey len = 0.207294 GB avg = 0.020 KB total ukey num = 0.010364724 Billion total entry num = 0.010364724 Billion write speed all = 0.022441312 MB/sec (with seq num) write speed all = 0.021232406 MB/sec (without seq num) 2024/11/23-03:55:32.244566 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243616: 932500 keys, 36499262 bytes, temperature: kWarm 2024/11/23-03:55:32.244626 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305332244595, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243616, "file_size": 36499262, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3771168700, "table_properties": {"data_size": 30261088, "index_size": 1774016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26110000, "raw_average_key_size": 28, "raw_value_size": 108107736, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932500, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305329, "oldest_key_time": 0, "file_creation_time": 1732305329, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243616, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:32.278483 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243617.sst) /dev/shm/ToplingTemp/Topling-401183-o5w4ZK 2024/11/23-03:55:33.154527 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243617.sst): first pass time = 0.88's, 153.215'MB/sec 2024/11/23-03:55:34.482416 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243617.sst): old prealloc_size = 147639500, real_size = 33616016 2024/11/23-03:55:34.482971 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243617.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930486 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0316 GB avg-val = 116.24 avg-zval = 33.95 2024/11/23-03:55:34.483676 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243617.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:35.369194 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243617.sst): second pass time = 1.33's, 81.501'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 42.093'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3381.002'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.308'MB/sec write SST data time = 0.13's, 284.915'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29931.488'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.501'MB/sec zip pipeline throughput = 81.501'MB/sec entries = 930486 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.24 avg-zval = 33.95 usrkeys = 930486 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.87 avg-zval = 33.95 TagRS{ kinds = RS_Key0_TagN bytes = 145480 } TagArray{ size = 747489 bytes = 2709664 } seq expand size = 5232423 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.3778 value = 3.4240 dict = 1.00 all = 3.5958 } Zip/UnZip{ index = 0.0964 value = 0.2921 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 1.362414 GB avg = 0.121 KB (by entry num) total key len = 0.316266 GB avg = 0.028 KB total ukey len = 0.225904 GB avg = 0.020 KB total ukey num = 0.011295210 Billion total entry num = 0.011295210 Billion write speed all = 0.024473311 MB/sec (with seq num) write speed all = 0.023155937 MB/sec (without seq num) 2024/11/23-03:55:35.405086 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243617: 930486 keys, 37326366 bytes, temperature: kWarm 2024/11/23-03:55:35.405153 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305335405120, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243617, "file_size": 37326366, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3771168954, "table_properties": {"data_size": 31590016, "index_size": 1793216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26053608, "raw_average_key_size": 28, "raw_value_size": 108164224, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930486, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305332, "oldest_key_time": 0, "file_creation_time": 1732305332, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243617, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:35.441388 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243618.sst) /dev/shm/ToplingTemp/Topling-401183-LCbDTJ 2024/11/23-03:55:36.310872 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243618.sst): first pass time = 0.87's, 154.373'MB/sec 2024/11/23-03:55:37.632175 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243618.sst): old prealloc_size = 147639500, real_size = 31789440 2024/11/23-03:55:37.632730 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243618.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927482 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0298 GB avg-val = 116.71 avg-zval = 32.12 2024/11/23-03:55:37.633671 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243618.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:38.514690 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243618.sst): second pass time = 1.32's, 81.970'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.801'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3317.717'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 279.853'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 22533.681'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.970'MB/sec zip pipeline throughput = 81.970'MB/sec entries = 927482 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.71 avg-zval = 32.12 usrkeys = 927482 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.71 avg-zval = 32.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927482 bytes = 3362144 } seq expand size = 6492374 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5136 value = 3.6333 dict = 1.00 all = 3.7277 } Zip/UnZip{ index = 0.0951 value = 0.2752 dict = 1.00 all = 0.2683 } ---------------------------- total value len = 1.477155 GB avg = 0.121 KB (by entry num) total key len = 0.342235 GB avg = 0.028 KB total ukey len = 0.244454 GB avg = 0.020 KB total ukey num = 0.012222692 Billion total entry num = 0.012222692 Billion write speed all = 0.026523495 MB/sec (with seq num) write speed all = 0.025098013 MB/sec (without seq num) 2024/11/23-03:55:38.545838 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243618: 927482 keys, 36006010 bytes, temperature: kWarm 2024/11/23-03:55:38.545901 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305338545868, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243618, "file_size": 36006010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3488176774, "largest_seqno": 3771168149, "table_properties": {"data_size": 29793008, "index_size": 1764352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25969496, "raw_average_key_size": 28, "raw_value_size": 108248324, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927482, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305335, "oldest_key_time": 0, "file_creation_time": 1732305335, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243618, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:38.584345 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243619.sst) /dev/shm/ToplingTemp/Topling-401183-7Wm47M 2024/11/23-03:55:39.509695 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243619.sst): first pass time = 0.93's, 145.050'MB/sec 2024/11/23-03:55:40.843899 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243619.sst): old prealloc_size = 147639500, real_size = 32861664 2024/11/23-03:55:40.844407 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243619.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932686 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 115.90 avg-zval = 33.07 2024/11/23-03:55:40.845254 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243619.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:55:41.776274 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243619.sst): second pass time = 1.33's, 81.068'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.784'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3674.756'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 291.193'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32383.753'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.068'MB/sec zip pipeline throughput = 81.068'MB/sec entries = 932686 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.90 avg-zval = 33.07 usrkeys = 932686 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.90 avg-zval = 33.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932686 bytes = 3381008 } seq expand size = 6528802 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4456 value = 3.5050 dict = 1.00 all = 3.6184 } Zip/UnZip{ index = 0.0957 value = 0.2853 dict = 1.00 all = 0.2764 } ---------------------------- total value len = 1.591786 GB avg = 0.121 KB (by entry num) total key len = 0.368351 GB avg = 0.028 KB total ukey len = 0.263108 GB avg = 0.020 KB total ukey num = 0.013155378 Billion total entry num = 0.013155378 Billion write speed all = 0.028573972 MB/sec (with seq num) write speed all = 0.027039788 MB/sec (without seq num) 2024/11/23-03:55:41.808786 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243619: 932686 keys, 37093270 bytes, temperature: kWarm 2024/11/23-03:55:41.808847 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305341808814, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243619, "file_size": 37093270, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3771168184, "table_properties": {"data_size": 30842512, "index_size": 1785792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26115208, "raw_average_key_size": 28, "raw_value_size": 108102548, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932686, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305338, "oldest_key_time": 0, "file_creation_time": 1732305338, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243619, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:41.849075 140024852538944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243620.sst) /dev/shm/ToplingTemp/Topling-401183-r6iDtL 2024/11/23-03:55:42.064890 140024852538944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243620.sst): first pass time = 0.22's, 251.588'MB/sec 2024/11/23-03:55:42.590402 140024852538944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243620.sst): old prealloc_size = 147639500, real_size = 15097952 2024/11/23-03:55:42.590671 140024852538944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243620.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 372787 prefix = 4 raw-key = 0.0060 GB zip-key = 0.0007 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0439 GB zip-val = 0.0143 GB avg-val = 117.63 avg-zval = 38.34 2024/11/23-03:55:42.591042 140024852538944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243620.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.054 GB 2024/11/23-03:55:42.950615 140024852538944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00/243620.sst): second pass time = 0.52's, 83.540'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.36's, 46.648'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2895.051'MB/sec (index lex order gen) rebuild zvType time = 0.31's, 0.303'MB/sec write SST data time = 0.05's, 318.728'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.442'MB, throughput = 13538.753'MB/sec zip my value time = 0.52's, unzip length = 0.044'GB zip my value throughput = 83.540'MB/sec zip pipeline throughput = 83.540'MB/sec entries = 372787 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.63 avg-zval = 38.34 usrkeys = 372787 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.63 avg-zval = 38.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 372787 bytes = 1351360 } seq expand size = 2609509 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0075 GB value = 0.0439 GB dict = 0.44 MB all = 0.0543 GB } __ZipSize{ index = 0.0007 GB value = 0.0143 GB dict = 0.44 MB all = 0.0168 GB } UnZip/Zip{ index = 10.4980 value = 3.0677 dict = 1.00 all = 3.2311 } Zip/UnZip{ index = 0.0953 value = 0.3260 dict = 1.00 all = 0.3095 } ---------------------------- total value len = 1.638246 GB avg = 0.121 KB (by entry num) total key len = 0.378789 GB avg = 0.028 KB total ukey len = 0.270563 GB avg = 0.020 KB total ukey num = 0.013528165 Billion total entry num = 0.013528165 Billion write speed all = 0.029402904 MB/sec (with seq num) write speed all = 0.027825272 MB/sec (without seq num) 2024/11/23-03:55:42.969169 140024852538944 [compaction_job.cc:1948] [default] [JOB 18771] Generated table #243620: 372787 keys, 16801643 bytes, temperature: kWarm 2024/11/23-03:55:42.969229 140024852538944 EVENT_LOG_v1 {"time_micros": 1732305342969196, "cf_name": "default", "job": 18771, "event": "table_file_creation", "file_number": 243620, "file_size": 16801643, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 3771168199, "table_properties": {"data_size": 14294352, "index_size": 710208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10438036, "raw_average_key_size": 28, "raw_value_size": 43850574, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 372787, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305341, "oldest_key_time": 0, "file_creation_time": 1732305341, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243620, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:55:42.985686 140024852538944 [compaction_job.cc:703] job-18771: subcompact[0], size: 0.531830 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18771/att-00 [243606,243607,243608,243609,243610,243611,243612,243613,243614,243615,243616,243617,243618,243619,243620] 2024/11/23-03:55:43.089961 140024852538944 (Original Log Time 2024/11/23-03:54:55.076850) EVENT_LOG_v1 {"time_micros": 1732305295076822, "job": 18771, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [243572], "files_L3": [243576, 243477, 243478, 243479, 243480, 243481, 243482, 243483, 243484, 243485, 243486, 243503, 243504, 243505, 243599], "score": 0.999384, "input_data_size": 554683636, "oldest_snapshot_seqno": 3771368969} 2024/11/23-03:55:43.089965 140024852538944 (Original Log Time 2024/11/23-03:55:43.089204) [compaction_job.cc:2022] [default] [JOB 18771] Compacted 1@2 + 15@3 files to L3 => 531830021 bytes 2024/11/23-03:55:43.089967 140024852538944 (Original Log Time 2024/11/23-03:55:43.089899) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.6 rd, 11.1 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(27.1, 501.9 +0.0 blob) out(507.2 +0.0 blob), read-write-amplify(38.3) write-amplify(18.7) OK, records in: 13613628, records dropped: 85463 output_compression: Snappy 2024/11/23-03:55:43.089973 140024852538944 (Original Log Time 2024/11/23-03:55:43.089943) EVENT_LOG_v1 {"time_micros": 1732305343089922, "cf": "default", "job": 18771, "event": "compaction_finished", "compaction_time_micros": 47908843, "compaction_time_cpu_micros": 21954490, "output_level": 3, "num_output_files": 15, "total_output_size": 531830021, "num_input_records": 13613628, "num_output_records": 13528165, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 351708533, "file_range_sync_nanos": 0, "file_fsync_nanos": 14023, "file_prepare_write_nanos": 37093, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-03:55:43.218742 140024852538944 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732305294,"executesMs":48149,"compactionJobId":18771,"attempt":0,"compactionInputRawBytes":1945473301,"compactionInputZipBytes":554683636,"compactionOutputRawBytes":0,"compactionOutputZipBytes":531830021,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c3ee251e465f9a1810a0c01f50c3b483-079c248ecd5a6953-00"}}, response = 2024/11/23-03:55:43.218813 140024852538944 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-18771/att-00: olev 3, work 48.149 s, result 4.936 ms, install 0.787 ms, input{raw 1.812 GiB zip 528.988 MiB}