2024/11/23-01:30:17.722055 140025230210624 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00: fork to child time = 0.026143 sec 2024/11/23-01:30:17.723113 140025230210624 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00 2024/11/23-01:30:17.731539 140025230210624 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00 2024/11/23-01:30:17.732994 140025230210624 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-01:30:17.733335 140025230210624 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-01:30:17.803868 140025230210624 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16417/att-00/MANIFEST-206347 2024/11/23-01:30:17.833763 140025230210624 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16417/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 238753, last_sequence is 3706941007, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 238567 2024/11/23-01:30:17.833776 140025230210624 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-01:30:17.833957 140025230210624 [version_set.cc:5721] Creating manifest 238753 2024/11/23-01:30:17.868811 140025230210624 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-01:30:17.918746 140025230210624 [compaction_job.cc:2437] [default] [JOB 16417] Compacting 1@2 + 16@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-01:30:17.918776 140025230210624 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 12290 Base level 2, inputs: [238712(27MB)], [238740(34MB) 238741(1734KB) 238665(34MB) 238666(35MB) 238667(34MB) 238668(33MB) 238669(36MB) 238670(36MB) 238671(34MB) 238672(11MB) 238636(35MB) 238637(35MB) 238638(33MB) 238639(36MB) 238640(35MB) 238742(35MB)] 2024/11/23-01:30:17.926811 140025230210624 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-01:30:17.926856 140025230210624 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-01:30:17.926886 140025230210624 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-01:30:17.926895 140025230210624 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-01:30:17.926910 140025230210624 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-01:30:18.431411 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238754.sst) /dev/shm/ToplingTemp/Topling-367583-iaNyg5 2024/11/23-01:30:19.093374 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238754.sst): first pass time = 0.66's, 202.771'MB/sec 2024/11/23-01:30:20.671694 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238754.sst): old prealloc_size = 147639500, real_size = 32077856 2024/11/23-01:30:20.672253 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238754.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928605 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0301 GB avg-val = 116.54 avg-zval = 32.39 2024/11/23-01:30:20.673056 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238754.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:21.677551 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238754.sst): second pass time = 1.58's, 68.603'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 35.856'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3284.224'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.264'MB/sec write SST data time = 0.13's, 284.233'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27018.197'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.603'MB/sec zip pipeline throughput = 94.520'MB/sec entries = 928605 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.54 avg-zval = 32.39 usrkeys = 928605 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.54 avg-zval = 32.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928605 bytes = 3134064 } seq expand size = 6500235 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5156 value = 3.5977 dict = 1.00 all = 3.7216 } Zip/UnZip{ index = 0.0951 value = 0.2780 dict = 1.00 all = 0.2687 } ---------------------------- total value len = 0.114717 GB avg = 0.124 KB (by entry num) total key len = 0.026001 GB avg = 0.028 KB total ukey len = 0.018572 GB avg = 0.020 KB total ukey num = 0.000928605 Billion total entry num = 0.000928605 Billion write speed all = 0.002350057 MB/sec (with seq num) write speed all = 0.002225992 MB/sec (without seq num) 2024/11/23-01:30:21.709893 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238754: 928605 keys, 36064956 bytes, temperature: kWarm 2024/11/23-01:30:21.709983 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296621709929, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238754, "file_size": 36064956, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3587898319, "largest_seqno": 3706704253, "table_properties": {"data_size": 30079376, "index_size": 1766144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26000940, "raw_average_key_size": 28, "raw_value_size": 108216914, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928605, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296618, "oldest_key_time": 0, "file_creation_time": 1732296618, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238754, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:21.753623 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238755.sst) /dev/shm/ToplingTemp/Topling-367583-HERIk6 2024/11/23-01:30:22.596137 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238755.sst): first pass time = 0.84's, 159.313'MB/sec 2024/11/23-01:30:24.037171 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238755.sst): old prealloc_size = 147639500, real_size = 35101408 2024/11/23-01:30:24.037816 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238755.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1082995 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 95.93 avg-zval = 30.23 2024/11/23-01:30:24.038014 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238755.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:24.674282 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238755.sst): second pass time = 1.44's, 72.157'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 56.944'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3342.078'MB/sec (index lex order gen) rebuild zvType time = 0.50's, 0.545'MB/sec write SST data time = 0.14's, 260.082'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.033'MB, throughput = 31216.029'MB/sec zip my value time = 1.44's, unzip length = 0.104'GB zip my value throughput = 72.157'MB/sec zip pipeline throughput = 72.157'MB/sec entries = 1082995 avg-key = 28.00 avg-zkey = 1.93 avg-val = 95.93 avg-zval = 30.23 usrkeys = 1082995 avg-key = 20.00 avg-zkey = 1.93 avg-val = 96.44 avg-zval = 30.23 TagRS{ kinds = RS_Key0_TagN bytes = 169320 } TagArray{ size = 78046 bytes = 234160 } seq expand size = 546322 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0217 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.03 MB all = 0.0363 GB } UnZip/Zip{ index = 10.3799 value = 3.1729 dict = 1.00 all = 3.6997 } Zip/UnZip{ index = 0.0963 value = 0.3152 dict = 1.00 all = 0.2703 } ---------------------------- total value len = 0.219157 GB avg = 0.109 KB (by entry num) total key len = 0.056325 GB avg = 0.028 KB total ukey len = 0.040232 GB avg = 0.020 KB total ukey num = 0.002011600 Billion total entry num = 0.002011600 Billion write speed all = 0.004600451 MB/sec (with seq num) write speed all = 0.004331707 MB/sec (without seq num) 2024/11/23-01:30:24.707839 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238755: 1082995 keys, 36278448 bytes, temperature: kWarm 2024/11/23-01:30:24.707919 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296624707876, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238755, "file_size": 36278448, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3706700881, "table_properties": {"data_size": 32743760, "index_size": 2086720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30323860, "raw_average_key_size": 28, "raw_value_size": 103894007, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1082995, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296621, "oldest_key_time": 0, "file_creation_time": 1732296621, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238755, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:24.762524 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238756.sst) /dev/shm/ToplingTemp/Topling-367583-hTDfU6 2024/11/23-01:30:25.676741 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238756.sst): first pass time = 0.91's, 146.821'MB/sec 2024/11/23-01:30:27.045759 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238756.sst): old prealloc_size = 147639500, real_size = 34986224 2024/11/23-01:30:27.046289 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238756.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923687 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0330 GB avg-val = 117.31 avg-zval = 35.72 2024/11/23-01:30:27.046808 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238756.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:27.877608 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238756.sst): second pass time = 1.37's, 79.241'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 45.622'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3530.177'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 293.458'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34184.482'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.241'MB/sec zip pipeline throughput = 79.241'MB/sec entries = 923687 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.31 avg-zval = 35.72 usrkeys = 923687 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.40 avg-zval = 35.72 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 540233 bytes = 1958352 } seq expand size = 3781631 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5129 value = 3.2837 dict = 1.00 all = 3.5367 } Zip/UnZip{ index = 0.0951 value = 0.3045 dict = 1.00 all = 0.2827 } ---------------------------- total value len = 0.331294 GB avg = 0.113 KB (by entry num) total key len = 0.082188 GB avg = 0.028 KB total ukey len = 0.058706 GB avg = 0.020 KB total ukey num = 0.002935287 Billion total entry num = 0.002935287 Billion write speed all = 0.006904620 MB/sec (with seq num) write speed all = 0.006512495 MB/sec (without seq num) 2024/11/23-01:30:27.914656 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238756: 923687 keys, 37950056 bytes, temperature: kWarm 2024/11/23-01:30:27.914721 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296627914688, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238756, "file_size": 37950056, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3706702693, "table_properties": {"data_size": 32997856, "index_size": 1757248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25863236, "raw_average_key_size": 28, "raw_value_size": 108354498, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923687, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296624, "oldest_key_time": 0, "file_creation_time": 1732296624, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238756, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:27.952308 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238757.sst) /dev/shm/ToplingTemp/Topling-367583-z82vr7 2024/11/23-01:30:28.857126 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238757.sst): first pass time = 0.90's, 148.344'MB/sec 2024/11/23-01:30:30.228497 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238757.sst): old prealloc_size = 147639500, real_size = 32274272 2024/11/23-01:30:30.229084 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238757.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931663 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 116.06 avg-zval = 32.49 2024/11/23-01:30:30.229834 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238757.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:31.160989 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238757.sst): second pass time = 1.37's, 78.926'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.021'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3137.906'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 284.140'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34354.996'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.926'MB/sec zip pipeline throughput = 78.926'MB/sec entries = 931663 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 32.49 usrkeys = 931663 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 32.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931663 bytes = 3260832 } seq expand size = 6521641 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5114 value = 3.5724 dict = 1.00 all = 3.6887 } Zip/UnZip{ index = 0.0951 value = 0.2799 dict = 1.00 all = 0.2711 } ---------------------------- total value len = 0.445947 GB avg = 0.115 KB (by entry num) total key len = 0.108275 GB avg = 0.028 KB total ukey len = 0.077339 GB avg = 0.020 KB total ukey num = 0.003866950 Billion total entry num = 0.003866950 Billion write speed all = 0.009254284 MB/sec (with seq num) write speed all = 0.008737727 MB/sec (without seq num) 2024/11/23-01:30:31.192134 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238757: 931663 keys, 36385872 bytes, temperature: kWarm 2024/11/23-01:30:31.192196 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296631192165, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238757, "file_size": 36385872, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3706704118, "table_properties": {"data_size": 30268496, "index_size": 1772672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26086564, "raw_average_key_size": 28, "raw_value_size": 108131294, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931663, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296627, "oldest_key_time": 0, "file_creation_time": 1732296627, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238757, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:31.234120 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238758.sst) /dev/shm/ToplingTemp/Topling-367583-5GPxl4 2024/11/23-01:30:32.072657 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238758.sst): first pass time = 0.84's, 160.068'MB/sec 2024/11/23-01:30:33.414861 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238758.sst): old prealloc_size = 147639500, real_size = 31802672 2024/11/23-01:30:33.415440 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238758.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920381 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0298 GB avg-val = 117.83 avg-zval = 32.40 2024/11/23-01:30:33.416224 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238758.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:34.295760 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238758.sst): second pass time = 1.34's, 80.836'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.867'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3143.333'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.307'MB/sec write SST data time = 0.13's, 276.669'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 31538.220'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.836'MB/sec zip pipeline throughput = 80.836'MB/sec entries = 920381 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.83 avg-zval = 32.40 usrkeys = 920381 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.83 avg-zval = 32.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920381 bytes = 3336400 } seq expand size = 6442667 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5132 value = 3.6365 dict = 1.00 all = 3.7284 } Zip/UnZip{ index = 0.0951 value = 0.2750 dict = 1.00 all = 0.2682 } ---------------------------- total value len = 0.560836 GB avg = 0.117 KB (by entry num) total key len = 0.134045 GB avg = 0.028 KB total ukey len = 0.095747 GB avg = 0.020 KB total ukey num = 0.004787331 Billion total entry num = 0.004787331 Billion write speed all = 0.011602398 MB/sec (with seq num) write speed all = 0.010962928 MB/sec (without seq num) 2024/11/23-01:30:34.331202 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238758: 920381 keys, 35999116 bytes, temperature: kWarm 2024/11/23-01:30:34.331264 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296634331233, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238758, "file_size": 35999116, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375334738, "largest_seqno": 3706704076, "table_properties": {"data_size": 29821472, "index_size": 1750912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25770668, "raw_average_key_size": 28, "raw_value_size": 108447082, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920381, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296631, "oldest_key_time": 0, "file_creation_time": 1732296631, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238758, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:34.380454 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238759.sst) /dev/shm/ToplingTemp/Topling-367583-iRFFY7 2024/11/23-01:30:35.329924 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238759.sst): first pass time = 0.95's, 141.367'MB/sec 2024/11/23-01:30:36.684444 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238759.sst): old prealloc_size = 147639500, real_size = 33805904 2024/11/23-01:30:36.684996 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238759.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930714 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0318 GB avg-val = 116.21 avg-zval = 34.16 2024/11/23-01:30:36.685767 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238759.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:37.617497 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238759.sst): second pass time = 1.35's, 79.893'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.761'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3360.901'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.13's, 287.255'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32305.185'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.893'MB/sec zip pipeline throughput = 79.893'MB/sec entries = 930714 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.21 avg-zval = 34.16 usrkeys = 930714 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.21 avg-zval = 34.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930714 bytes = 3373856 } seq expand size = 6514998 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.4426 value = 3.4022 dict = 1.00 all = 3.5291 } Zip/UnZip{ index = 0.0958 value = 0.2939 dict = 1.00 all = 0.2834 } ---------------------------- total value len = 0.675509 GB avg = 0.118 KB (by entry num) total key len = 0.160105 GB avg = 0.028 KB total ukey len = 0.114361 GB avg = 0.020 KB total ukey num = 0.005718045 Billion total entry num = 0.005718045 Billion write speed all = 0.013951431 MB/sec (with seq num) write speed all = 0.013187683 MB/sec (without seq num) 2024/11/23-01:30:37.652795 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238759: 930714 keys, 38031420 bytes, temperature: kWarm 2024/11/23-01:30:37.652865 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296637652829, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238759, "file_size": 38031420, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3706702734, "table_properties": {"data_size": 31790528, "index_size": 1782528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26059992, "raw_average_key_size": 28, "raw_value_size": 108157752, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930714, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296634, "oldest_key_time": 0, "file_creation_time": 1732296634, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238759, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:37.699857 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238760.sst) /dev/shm/ToplingTemp/Topling-367583-xVPy24 2024/11/23-01:30:38.618070 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238760.sst): first pass time = 0.92's, 146.180'MB/sec 2024/11/23-01:30:40.009727 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238760.sst): old prealloc_size = 147639500, real_size = 34318800 2024/11/23-01:30:40.010259 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238760.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926847 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0323 GB avg-val = 116.81 avg-zval = 34.86 2024/11/23-01:30:40.011034 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238760.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:40.926727 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238760.sst): second pass time = 1.39's, 77.836'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 42.021'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3381.769'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 296.744'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33789.737'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.836'MB/sec zip pipeline throughput = 77.836'MB/sec entries = 926847 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.81 avg-zval = 34.86 usrkeys = 926847 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.81 avg-zval = 34.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926847 bytes = 3359840 } seq expand size = 6487929 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.4431 value = 3.3507 dict = 1.00 all = 3.4832 } Zip/UnZip{ index = 0.0958 value = 0.2984 dict = 1.00 all = 0.2871 } ---------------------------- total value len = 0.790263 GB avg = 0.119 KB (by entry num) total key len = 0.186057 GB avg = 0.028 KB total ukey len = 0.132898 GB avg = 0.020 KB total ukey num = 0.006644892 Billion total entry num = 0.006644892 Billion write speed all = 0.016299756 MB/sec (with seq num) write speed all = 0.015412259 MB/sec (without seq num) 2024/11/23-01:30:40.966055 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238760: 926847 keys, 38532672 bytes, temperature: kWarm 2024/11/23-01:30:40.966118 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296640966087, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238760, "file_size": 38532672, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3339428666, "largest_seqno": 3706703895, "table_properties": {"data_size": 32311872, "index_size": 1775040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951716, "raw_average_key_size": 28, "raw_value_size": 108266074, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926847, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296637, "oldest_key_time": 0, "file_creation_time": 1732296637, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238760, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:41.003188 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238761.sst) /dev/shm/ToplingTemp/Topling-367583-KQxpd6 2024/11/23-01:30:41.883926 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238761.sst): first pass time = 0.88's, 152.400'MB/sec 2024/11/23-01:30:43.227467 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238761.sst): old prealloc_size = 147639500, real_size = 31005536 2024/11/23-01:30:43.228053 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238761.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926495 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 116.87 avg-zval = 31.30 2024/11/23-01:30:43.229491 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238761.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:44.108938 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238761.sst): second pass time = 1.34's, 80.655'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 39.824'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3147.649'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 267.528'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 23694.092'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.655'MB/sec zip pipeline throughput = 80.655'MB/sec entries = 926495 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.87 avg-zval = 31.30 usrkeys = 926495 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.87 avg-zval = 31.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926495 bytes = 3242752 } seq expand size = 6485465 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4444 value = 3.7337 dict = 1.00 all = 3.8235 } Zip/UnZip{ index = 0.0957 value = 0.2678 dict = 1.00 all = 0.2615 } ---------------------------- total value len = 0.905024 GB avg = 0.120 KB (by entry num) total key len = 0.211999 GB avg = 0.028 KB total ukey len = 0.151428 GB avg = 0.020 KB total ukey num = 0.007571387 Billion total entry num = 0.007571387 Billion write speed all = 0.018647820 MB/sec (with seq num) write speed all = 0.017636633 MB/sec (without seq num) 2024/11/23-01:30:44.140700 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238761: 926495 keys, 35102978 bytes, temperature: kWarm 2024/11/23-01:30:44.140760 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296644140728, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238761, "file_size": 35102978, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3531377150, "largest_seqno": 3706704329, "table_properties": {"data_size": 28999568, "index_size": 1774144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941860, "raw_average_key_size": 28, "raw_value_size": 108275946, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926495, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296640, "oldest_key_time": 0, "file_creation_time": 1732296640, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238761, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:44.202057 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238762.sst) /dev/shm/ToplingTemp/Topling-367583-z7WuZ6 2024/11/23-01:30:45.278853 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238762.sst): first pass time = 1.08's, 124.650'MB/sec 2024/11/23-01:30:46.640126 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238762.sst): old prealloc_size = 147639500, real_size = 32523072 2024/11/23-01:30:46.640669 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238762.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926694 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 116.84 avg-zval = 32.94 2024/11/23-01:30:46.641438 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238762.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:47.546716 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238762.sst): second pass time = 1.36's, 79.576'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.395'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3386.462'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.299'MB/sec write SST data time = 0.13's, 278.126'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33000.519'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.576'MB/sec zip pipeline throughput = 79.576'MB/sec entries = 926694 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.84 avg-zval = 32.94 usrkeys = 926694 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.84 avg-zval = 32.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926694 bytes = 3243440 } seq expand size = 6486858 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5138 value = 3.5465 dict = 1.00 all = 3.6651 } Zip/UnZip{ index = 0.0951 value = 0.2820 dict = 1.00 all = 0.2728 } ---------------------------- total value len = 1.019782 GB avg = 0.120 KB (by entry num) total key len = 0.237946 GB avg = 0.028 KB total ukey len = 0.169962 GB avg = 0.020 KB total ukey num = 0.008498081 Billion total entry num = 0.008498081 Billion write speed all = 0.020995567 MB/sec (with seq num) write speed all = 0.019860682 MB/sec (without seq num) 2024/11/23-01:30:47.578797 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238762: 926694 keys, 36620962 bytes, temperature: kWarm 2024/11/23-01:30:47.578860 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296647578829, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238762, "file_size": 36620962, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3706702800, "table_properties": {"data_size": 30528368, "index_size": 1762816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947432, "raw_average_key_size": 28, "raw_value_size": 108270336, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926694, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296644, "oldest_key_time": 0, "file_creation_time": 1732296644, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238762, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:47.615910 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238763.sst) /dev/shm/ToplingTemp/Topling-367583-H00ou8 2024/11/23-01:30:48.551183 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238763.sst): first pass time = 0.94's, 143.512'MB/sec 2024/11/23-01:30:49.892571 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238763.sst): old prealloc_size = 147639500, real_size = 32176288 2024/11/23-01:30:49.893127 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238763.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926617 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0302 GB avg-val = 116.85 avg-zval = 32.57 2024/11/23-01:30:49.893917 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238763.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:50.784289 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238763.sst): second pass time = 1.34's, 80.759'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.809'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3222.264'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 275.567'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35255.564'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.759'MB/sec zip pipeline throughput = 80.759'MB/sec entries = 926617 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.85 avg-zval = 32.57 usrkeys = 926617 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.85 avg-zval = 32.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926617 bytes = 3359008 } seq expand size = 6486319 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5137 value = 3.5873 dict = 1.00 all = 3.6883 } Zip/UnZip{ index = 0.0951 value = 0.2788 dict = 1.00 all = 0.2711 } ---------------------------- total value len = 1.134541 GB avg = 0.120 KB (by entry num) total key len = 0.263892 GB avg = 0.028 KB total ukey len = 0.188494 GB avg = 0.020 KB total ukey num = 0.009424698 Billion total entry num = 0.009424698 Billion write speed all = 0.023343115 MB/sec (with seq num) write speed all = 0.022084553 MB/sec (without seq num) 2024/11/23-01:30:50.818909 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238763: 926617 keys, 36389898 bytes, temperature: kWarm 2024/11/23-01:30:50.818974 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296650818941, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238763, "file_size": 36389898, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3706703981, "table_properties": {"data_size": 30181776, "index_size": 1762688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25945276, "raw_average_key_size": 28, "raw_value_size": 108272566, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926617, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296647, "oldest_key_time": 0, "file_creation_time": 1732296647, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238763, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:50.860826 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238764.sst) /dev/shm/ToplingTemp/Topling-367583-VcKsl8 2024/11/23-01:30:51.689166 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238764.sst): first pass time = 0.83's, 162.040'MB/sec 2024/11/23-01:30:53.041768 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238764.sst): old prealloc_size = 147639500, real_size = 34853984 2024/11/23-01:30:53.042356 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238764.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927394 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0328 GB avg-val = 116.73 avg-zval = 35.41 2024/11/23-01:30:53.043027 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238764.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:53.928258 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238764.sst): second pass time = 1.35's, 80.091'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.493'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.349'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.308'MB/sec write SST data time = 0.13's, 289.001'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35300.281'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.091'MB/sec zip pipeline throughput = 80.091'MB/sec entries = 927394 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.73 avg-zval = 35.41 usrkeys = 927394 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.34 avg-zval = 35.41 TagRS{ kinds = RS_Key0_TagN bytes = 145000 } TagArray{ size = 744397 bytes = 2698448 } seq expand size = 5210779 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.3793 value = 3.2968 dict = 1.00 all = 3.4812 } Zip/UnZip{ index = 0.0963 value = 0.3033 dict = 1.00 all = 0.2873 } ---------------------------- total value len = 1.248002 GB avg = 0.121 KB (by entry num) total key len = 0.289859 GB avg = 0.028 KB total ukey len = 0.207042 GB avg = 0.020 KB total ukey num = 0.010352092 Billion total entry num = 0.010352092 Billion write speed all = 0.025669159 MB/sec (with seq num) write speed all = 0.024286826 MB/sec (without seq num) 2024/11/23-01:30:53.962073 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238764: 927394 keys, 38555270 bytes, temperature: kWarm 2024/11/23-01:30:53.962143 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296653962112, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238764, "file_size": 38555270, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3706702940, "table_properties": {"data_size": 32834960, "index_size": 1787008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25967032, "raw_average_key_size": 28, "raw_value_size": 108250784, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927394, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296650, "oldest_key_time": 0, "file_creation_time": 1732296650, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238764, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:54.001240 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238765.sst) /dev/shm/ToplingTemp/Topling-367583-GCwF94 2024/11/23-01:30:54.976181 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238765.sst): first pass time = 0.97's, 137.673'MB/sec 2024/11/23-01:30:56.334152 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238765.sst): old prealloc_size = 147639500, real_size = 32838224 2024/11/23-01:30:56.334722 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238765.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931996 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.01 avg-zval = 33.08 2024/11/23-01:30:56.335490 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238765.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:30:57.297050 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238765.sst): second pass time = 1.36's, 79.658'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.376'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3233.544'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.13's, 283.321'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34133.853'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.658'MB/sec zip pipeline throughput = 79.658'MB/sec entries = 931996 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.01 avg-zval = 33.08 usrkeys = 931996 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.01 avg-zval = 33.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931996 bytes = 3262000 } seq expand size = 6523972 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5163 value = 3.5067 dict = 1.00 all = 3.6323 } Zip/UnZip{ index = 0.0951 value = 0.2852 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 1.362648 GB avg = 0.121 KB (by entry num) total key len = 0.315954 GB avg = 0.028 KB total ukey len = 0.225682 GB avg = 0.020 KB total ukey num = 0.011284088 Billion total entry num = 0.011284088 Billion write speed all = 0.028016772 MB/sec (with seq num) write speed all = 0.026510072 MB/sec (without seq num) 2024/11/23-01:30:57.331185 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238765: 931996 keys, 36951226 bytes, temperature: kWarm 2024/11/23-01:30:57.331247 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296657331216, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238765, "file_size": 36951226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3706704365, "table_properties": {"data_size": 30832576, "index_size": 1772480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26095888, "raw_average_key_size": 28, "raw_value_size": 108121956, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931996, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296653, "oldest_key_time": 0, "file_creation_time": 1732296653, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238765, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:30:57.370278 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238766.sst) /dev/shm/ToplingTemp/Topling-367583-aYFii6 2024/11/23-01:30:58.202470 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238766.sst): first pass time = 0.83's, 161.290'MB/sec 2024/11/23-01:30:59.519841 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238766.sst): old prealloc_size = 147639500, real_size = 32046640 2024/11/23-01:30:59.520414 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238766.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921819 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0301 GB avg-val = 117.60 avg-zval = 32.61 2024/11/23-01:30:59.521157 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238766.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:31:00.404550 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238766.sst): second pass time = 1.32's, 82.332'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.841'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3181.028'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.306'MB/sec write SST data time = 0.13's, 274.948'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32431.235'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.332'MB/sec zip pipeline throughput = 82.332'MB/sec entries = 921819 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.60 avg-zval = 32.61 usrkeys = 921819 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.60 avg-zval = 32.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921819 bytes = 3226384 } seq expand size = 6452733 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5115 value = 3.6061 dict = 1.00 all = 3.7146 } Zip/UnZip{ index = 0.0951 value = 0.2773 dict = 1.00 all = 0.2692 } ---------------------------- total value len = 1.477508 GB avg = 0.121 KB (by entry num) total key len = 0.341765 GB avg = 0.028 KB total ukey len = 0.244118 GB avg = 0.020 KB total ukey num = 0.012205907 Billion total entry num = 0.012205907 Billion write speed all = 0.030363063 MB/sec (with seq num) write speed all = 0.028733362 MB/sec (without seq num) 2024/11/23-01:31:00.439520 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238766: 921819 keys, 36132024 bytes, temperature: kWarm 2024/11/23-01:31:00.439586 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296660439551, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238766, "file_size": 36132024, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 3706704385, "table_properties": {"data_size": 30062112, "index_size": 1753920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25810932, "raw_average_key_size": 28, "raw_value_size": 108406842, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921819, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296657, "oldest_key_time": 0, "file_creation_time": 1732296657, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238766, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:31:00.478990 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238767.sst) /dev/shm/ToplingTemp/Topling-367583-Y40Jr6 2024/11/23-01:31:01.278394 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238767.sst): first pass time = 0.80's, 167.908'MB/sec 2024/11/23-01:31:02.605180 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238767.sst): old prealloc_size = 147639500, real_size = 32422688 2024/11/23-01:31:02.605723 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238767.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930237 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.28 avg-zval = 32.69 2024/11/23-01:31:02.606476 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238767.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-01:31:03.545256 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238767.sst): second pass time = 1.33's, 81.575'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 38.736'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3430.380'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.288'MB/sec write SST data time = 0.13's, 277.697'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32486.431'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.575'MB/sec zip pipeline throughput = 81.575'MB/sec entries = 930237 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.28 avg-zval = 32.69 usrkeys = 930237 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.28 avg-zval = 32.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930237 bytes = 3139568 } seq expand size = 6511659 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4440 value = 3.5573 dict = 1.00 all = 3.6859 } Zip/UnZip{ index = 0.0957 value = 0.2811 dict = 1.00 all = 0.2713 } ---------------------------- total value len = 1.592190 GB avg = 0.121 KB (by entry num) total key len = 0.367812 GB avg = 0.028 KB total ukey len = 0.262723 GB avg = 0.020 KB total ukey num = 0.013136144 Billion total entry num = 0.013136144 Billion write speed all = 0.032710076 MB/sec (with seq num) write speed all = 0.030956265 MB/sec (without seq num) 2024/11/23-01:31:03.580302 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238767: 930237 keys, 36414150 bytes, temperature: kWarm 2024/11/23-01:31:03.580367 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296663580334, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238767, "file_size": 36414150, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3604582272, "largest_seqno": 3706704397, "table_properties": {"data_size": 30408592, "index_size": 1781376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26046636, "raw_average_key_size": 28, "raw_value_size": 108171138, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930237, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296660, "oldest_key_time": 0, "file_creation_time": 1732296660, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238767, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:31:03.621869 140025230210624 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238768.sst) /dev/shm/ToplingTemp/Topling-367583-uIc0g6 2024/11/23-01:31:03.897993 140025230210624 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238768.sst): first pass time = 0.28's, 242.068'MB/sec 2024/11/23-01:31:04.559869 140025230210624 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238768.sst): old prealloc_size = 147639500, real_size = 17945600 2024/11/23-01:31:04.560192 140025230210624 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238768.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 460479 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0539 GB zip-val = 0.0170 GB avg-val = 117.13 avg-zval = 36.82 2024/11/23-01:31:04.560600 140025230210624 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238768.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.067 GB 2024/11/23-01:31:05.018792 140025230210624 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00/238768.sst): second pass time = 0.66's, 81.573'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.46's, 43.561'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2922.940'MB/sec (index lex order gen) rebuild zvType time = 0.39's, 0.292'MB/sec write SST data time = 0.06's, 309.671'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.544'MB, throughput = 13445.326'MB/sec zip my value time = 0.66's, unzip length = 0.054'GB zip my value throughput = 81.573'MB/sec zip pipeline throughput = 81.573'MB/sec entries = 460479 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.13 avg-zval = 36.82 usrkeys = 460479 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.13 avg-zval = 36.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 460479 bytes = 1611696 } seq expand size = 3223353 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0092 GB value = 0.0539 GB dict = 0.54 MB all = 0.0668 GB } __ZipSize{ index = 0.0009 GB value = 0.0170 GB dict = 0.54 MB all = 0.0200 GB } UnZip/Zip{ index = 10.5044 value = 3.1815 dict = 1.00 all = 3.3432 } Zip/UnZip{ index = 0.0952 value = 0.3143 dict = 1.00 all = 0.2991 } ---------------------------- total value len = 1.649351 GB avg = 0.121 KB (by entry num) total key len = 0.380705 GB avg = 0.028 KB total ukey len = 0.271932 GB avg = 0.020 KB total ukey num = 0.013596623 Billion total entry num = 0.013596623 Billion write speed all = 0.033878361 MB/sec (with seq num) write speed all = 0.032063116 MB/sec (without seq num) 2024/11/23-01:31:05.043645 140025230210624 [compaction_job.cc:1948] [default] [JOB 16417] Generated table #238768: 460479 keys, 19989803 bytes, temperature: kWarm 2024/11/23-01:31:05.043710 140025230210624 EVENT_LOG_v1 {"time_micros": 1732296665043678, "cf_name": "default", "job": 16417, "event": "table_file_creation", "file_number": 238768, "file_size": 19989803, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3706702185, "table_properties": {"data_size": 16953584, "index_size": 876736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12893412, "raw_average_key_size": 28, "raw_value_size": 53937326, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 460479, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732296663, "oldest_key_time": 0, "file_creation_time": 1732296663, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 238768, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-01:31:05.062989 140025230210624 [compaction_job.cc:703] job-16417: subcompact[0], size: 0.535399 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-16417/att-00 [238754,238755,238756,238757,238758,238759,238760,238761,238762,238763,238764,238765,238766,238767,238768] 2024/11/23-01:31:05.156205 140025230210624 (Original Log Time 2024/11/23-01:30:17.918876) EVENT_LOG_v1 {"time_micros": 1732296617918814, "job": 16417, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [238712], "files_L3": [238740, 238741, 238665, 238666, 238667, 238668, 238669, 238670, 238671, 238672, 238636, 238637, 238638, 238639, 238640, 238742], "score": 0.997547, "input_data_size": 559394555, "oldest_snapshot_seqno": 3706940971} 2024/11/23-01:31:05.156208 140025230210624 (Original Log Time 2024/11/23-01:31:05.155415) [compaction_job.cc:2022] [default] [JOB 16417] Compacted 1@2 + 16@3 files to L3 => 535398851 bytes 2024/11/23-01:31:05.156209 140025230210624 (Original Log Time 2024/11/23-01:31:05.156140) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.9 rd, 11.4 wr, level 3, files in(1, 16) out(15 +0 blob) MB in(27.2, 506.3 +0.0 blob) out(510.6 +0.0 blob), read-write-amplify(38.4) write-amplify(18.8) OK, records in: 13676739, records dropped: 80116 output_compression: Snappy 2024/11/23-01:31:05.156216 140025230210624 (Original Log Time 2024/11/23-01:31:05.156184) EVENT_LOG_v1 {"time_micros": 1732296665156165, "cf": "default", "job": 16417, "event": "compaction_finished", "compaction_time_micros": 47144123, "compaction_time_cpu_micros": 21783700, "output_level": 3, "num_output_files": 15, "total_output_size": 535398851, "num_input_records": 13676739, "num_output_records": 13596623, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 333971463, "file_range_sync_nanos": 0, "file_fsync_nanos": 12987, "file_prepare_write_nanos": 33854, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-01:31:05.283891 140025230210624 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732296617,"executesMs":47428,"compactionJobId":16417,"attempt":0,"compactionInputRawBytes":1957256431,"compactionInputZipBytes":559394555,"compactionOutputRawBytes":0,"compactionOutputZipBytes":535398851,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-3808056f652b5de1d74b1e7e160441e7-f6c729128d24f3ec-00"}}, response = 2024/11/23-01:31:05.283978 140025230210624 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-16417/att-00: olev 3, work 47.428 s, result 4.924 ms, install 0.820 ms, input{raw 1.823 GiB zip 533.480 MiB}