2024/11/23-14:37:46.670157 139957248734784 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00: fork to child time = 0.021669 sec 2024/11/23-14:37:46.671100 139957248734784 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00 2024/11/23-14:37:46.678359 139957248734784 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00 2024/11/23-14:37:46.679260 139957248734784 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:37:46.679507 139957248734784 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:37:46.737314 139957248734784 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28087/att-00/MANIFEST-206347 2024/11/23-14:37:46.775396 139957248734784 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28087/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 263843, last_sequence is 4030928892, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 263683 2024/11/23-14:37:46.775403 139957248734784 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:37:46.775523 139957248734784 [version_set.cc:5721] Creating manifest 263843 2024/11/23-14:37:46.813240 139957248734784 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:37:46.835902 139957248734784 [compaction_job.cc:2437] [default] [JOB 28087] Compacting 1@2 + 26@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-14:37:46.835924 139957248734784 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20983 Base level 2, inputs: [263823(27MB)], [263726(37MB) 263782(40MB) 263784(36MB) 263785(34MB) 263786(35MB) 263787(35MB) 263788(34MB) 263789(34MB) 263790(34MB) 263791(36MB) 263792(35MB) 263793(35MB) 263794(35MB) 263795(36MB) 263796(35MB) 263797(36MB) 263798(37MB) 263799(35MB) 263800(34MB) 263801(37MB) 263802(33MB) 263803(37MB) 263804(34MB) 263805(34MB) 263806(36MB) 263838(39MB)] 2024/11/23-14:37:46.843753 139957248734784 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:37:46.843778 139957248734784 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:37:46.843794 139957248734784 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:37:46.843800 139957248734784 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:37:46.843810 139957248734784 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:37:47.446686 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263844.sst) /dev/shm/ToplingTemp/Topling-77035-qrROWd 2024/11/23-14:37:48.511687 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263844.sst): first pass time = 1.06's, 126.033'MB/sec 2024/11/23-14:37:50.532166 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263844.sst): old prealloc_size = 147639500, real_size = 35192016 2024/11/23-14:37:50.532731 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263844.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929411 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0332 GB avg-val = 116.41 avg-zval = 35.72 2024/11/23-14:37:50.533523 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263844.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:51.770500 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263844.sst): second pass time = 2.02's, 53.575'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.24's, 31.644'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3230.854'MB/sec (index lex order gen) rebuild zvType time = 1.11's, 0.209'MB/sec write SST data time = 0.13's, 311.289'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30136.810'MB/sec zip my value time = 2.02's, unzip length = 0.108'GB zip my value throughput = 53.575'MB/sec zip pipeline throughput = 83.088'MB/sec entries = 929411 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.41 avg-zval = 35.72 usrkeys = 929411 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.41 avg-zval = 35.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929411 bytes = 3136784 } seq expand size = 6505877 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5823 value = 3.2586 dict = 1.00 all = 3.4252 } Zip/UnZip{ index = 0.0945 value = 0.3069 dict = 1.00 all = 0.2920 } ---------------------------- total value len = 8.963320 GB avg = 0.168 KB (by entry num) total key len = 1.466845 GB avg = 0.028 KB total ukey len = 1.016466 GB avg = 0.015 KB total ukey num = 0.066479846 Billion total entry num = 0.053287575 Billion write speed all = 0.507314682 MB/sec (with seq num) write speed all = 0.486579770 MB/sec (without seq num) 2024/11/23-14:37:51.819829 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263844: 929411 keys, 39185380 bytes, temperature: kWarm 2024/11/23-14:37:51.819922 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343871819865, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263844, "file_size": 39185380, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3934836997, "largest_seqno": 4030479700, "table_properties": {"data_size": 33202944, "index_size": 1756544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26023508, "raw_average_key_size": 28, "raw_value_size": 108194290, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929411, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343867, "oldest_key_time": 0, "file_creation_time": 1732343867, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263844, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:51.858271 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263845.sst) /dev/shm/ToplingTemp/Topling-77035-9qOrwe 2024/11/23-14:37:52.894085 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263845.sst): first pass time = 1.04's, 129.583'MB/sec 2024/11/23-14:37:54.539943 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263845.sst): old prealloc_size = 147639500, real_size = 38268304 2024/11/23-14:37:54.540530 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263845.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923757 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0363 GB avg-val = 117.30 avg-zval = 39.27 2024/11/23-14:37:54.541281 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263845.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:55.514253 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263845.sst): second pass time = 1.65's, 65.860'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 43.121'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3163.848'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.273'MB/sec write SST data time = 0.13's, 332.299'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32587.732'MB/sec zip my value time = 1.65's, unzip length = 0.108'GB zip my value throughput = 65.860'MB/sec zip pipeline throughput = 65.860'MB/sec entries = 923757 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.30 avg-zval = 39.27 usrkeys = 923757 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.30 avg-zval = 39.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923757 bytes = 2886752 } seq expand size = 6466299 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0363 GB dict = 1.08 MB all = 0.0420 GB } UnZip/Zip{ index = 10.5133 value = 2.9866 dict = 1.00 all = 3.1947 } Zip/UnZip{ index = 0.0951 value = 0.3348 dict = 1.00 all = 0.3130 } ---------------------------- total value len = 9.078139 GB avg = 0.167 KB (by entry num) total key len = 1.492710 GB avg = 0.028 KB total ukey len = 1.034941 GB avg = 0.015 KB total ukey num = 0.067403603 Billion total entry num = 0.054211332 Billion write speed all = 0.514063835 MB/sec (with seq num) write speed all = 0.492973316 MB/sec (without seq num) 2024/11/23-14:37:55.564886 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263845: 923757 keys, 42012142 bytes, temperature: kWarm 2024/11/23-14:37:55.564953 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343875564918, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263845, "file_size": 42012142, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777827, "largest_seqno": 4030479989, "table_properties": {"data_size": 36279872, "index_size": 1757312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25865196, "raw_average_key_size": 28, "raw_value_size": 108352606, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923757, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343871, "oldest_key_time": 0, "file_creation_time": 1732343871, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263845, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:55.606614 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263846.sst) /dev/shm/ToplingTemp/Topling-77035-3cl6Wf 2024/11/23-14:37:56.778589 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263846.sst): first pass time = 1.17's, 114.526'MB/sec 2024/11/23-14:37:58.144768 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263846.sst): old prealloc_size = 147639500, real_size = 35199952 2024/11/23-14:37:58.145464 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263846.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925138 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0332 GB avg-val = 117.08 avg-zval = 35.88 2024/11/23-14:37:58.146335 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263846.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:37:59.216673 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263846.sst): second pass time = 1.37's, 79.332'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 36.770'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2629.688'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.251'MB/sec write SST data time = 0.15's, 260.607'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37806.163'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.332'MB/sec zip pipeline throughput = 79.332'MB/sec entries = 925138 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.08 avg-zval = 35.88 usrkeys = 925138 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.08 avg-zval = 35.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925138 bytes = 3353648 } seq expand size = 6475966 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4442 value = 3.2628 dict = 1.00 all = 3.4054 } Zip/UnZip{ index = 0.0957 value = 0.3065 dict = 1.00 all = 0.2936 } ---------------------------- total value len = 9.192929 GB avg = 0.167 KB (by entry num) total key len = 1.518614 GB avg = 0.028 KB total ukey len = 1.053444 GB avg = 0.015 KB total ukey num = 0.068328741 Billion total entry num = 0.055136470 Billion write speed all = 0.520812045 MB/sec (with seq num) write speed all = 0.499365470 MB/sec (without seq num) 2024/11/23-14:37:59.265642 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263846: 925138 keys, 39412772 bytes, temperature: kWarm 2024/11/23-14:37:59.265716 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343879265681, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263846, "file_size": 39412772, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4030479833, "table_properties": {"data_size": 33196928, "index_size": 1771584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903864, "raw_average_key_size": 28, "raw_value_size": 108313908, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925138, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343875, "oldest_key_time": 0, "file_creation_time": 1732343875, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263846, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:37:59.301043 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263847.sst) /dev/shm/ToplingTemp/Topling-77035-9ZUjBf 2024/11/23-14:38:00.552206 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263847.sst): first pass time = 1.25's, 107.277'MB/sec 2024/11/23-14:38:02.015434 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263847.sst): old prealloc_size = 147639500, real_size = 31892928 2024/11/23-14:38:02.016229 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263847.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931642 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0299 GB avg-val = 116.07 avg-zval = 32.07 2024/11/23-14:38:02.017192 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263847.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:02.924820 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263847.sst): second pass time = 1.46's, 73.946'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 39.721'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2310.283'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.16's, 231.792'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38039.801'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 73.946'MB/sec zip pipeline throughput = 73.946'MB/sec entries = 931642 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.07 avg-zval = 32.07 usrkeys = 931642 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.07 avg-zval = 32.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931642 bytes = 3377216 } seq expand size = 6521494 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4459 value = 3.6193 dict = 1.00 all = 3.7158 } Zip/UnZip{ index = 0.0957 value = 0.2763 dict = 1.00 all = 0.2691 } ---------------------------- total value len = 9.307582 GB avg = 0.166 KB (by entry num) total key len = 1.544700 GB avg = 0.028 KB total ukey len = 1.072077 GB avg = 0.015 KB total ukey num = 0.069260383 Billion total entry num = 0.056068112 Billion write speed all = 0.527559894 MB/sec (with seq num) write speed all = 0.505754866 MB/sec (without seq num) 2024/11/23-14:38:02.970091 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263847: 931642 keys, 36121302 bytes, temperature: kWarm 2024/11/23-14:38:02.970155 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343882970123, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263847, "file_size": 36121302, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 4012776122, "table_properties": {"data_size": 29876080, "index_size": 1783744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26085976, "raw_average_key_size": 28, "raw_value_size": 108131812, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931642, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343879, "oldest_key_time": 0, "file_creation_time": 1732343879, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263847, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:03.009223 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263848.sst) /dev/shm/ToplingTemp/Topling-77035-NA0I6e 2024/11/23-14:38:04.127265 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263848.sst): first pass time = 1.12's, 120.052'MB/sec 2024/11/23-14:38:05.704704 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263848.sst): old prealloc_size = 147639500, real_size = 32052720 2024/11/23-14:38:05.705422 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263848.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931352 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0300 GB avg-val = 116.11 avg-zval = 32.25 2024/11/23-14:38:05.706341 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263848.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:06.557632 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263848.sst): second pass time = 1.58's, 68.637'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.672'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2558.068'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 272.518'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34637.029'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.637'MB/sec zip pipeline throughput = 68.637'MB/sec entries = 931352 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.11 avg-zval = 32.25 usrkeys = 931352 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.11 avg-zval = 32.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931352 bytes = 3492592 } seq expand size = 6519464 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4472 value = 3.6003 dict = 1.00 all = 3.6877 } Zip/UnZip{ index = 0.0957 value = 0.2778 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 9.422241 GB avg = 0.165 KB (by entry num) total key len = 1.570777 GB avg = 0.028 KB total ukey len = 1.090704 GB avg = 0.016 KB total ukey num = 0.070191735 Billion total entry num = 0.056999464 Billion write speed all = 0.534307169 MB/sec (with seq num) write speed all = 0.512143851 MB/sec (without seq num) 2024/11/23-14:38:06.601772 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263848: 931352 keys, 36396352 bytes, temperature: kWarm 2024/11/23-14:38:06.601840 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343886601807, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263848, "file_size": 36396352, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012776133, "table_properties": {"data_size": 30036704, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26077856, "raw_average_key_size": 28, "raw_value_size": 108139968, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931352, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343882, "oldest_key_time": 0, "file_creation_time": 1732343882, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263848, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:06.642624 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263849.sst) /dev/shm/ToplingTemp/Topling-77035-PaoRSb 2024/11/23-14:38:07.715005 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263849.sst): first pass time = 1.07's, 125.165'MB/sec 2024/11/23-14:38:09.225102 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263849.sst): old prealloc_size = 147639500, real_size = 32958784 2024/11/23-14:38:09.225774 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263849.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930254 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.28 avg-zval = 33.26 2024/11/23-14:38:09.226673 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263849.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:10.121487 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263849.sst): second pass time = 1.51's, 71.672'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 41.482'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2747.832'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.15's, 251.857'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 44591.751'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.672'MB/sec zip pipeline throughput = 71.672'MB/sec entries = 930254 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.28 avg-zval = 33.26 usrkeys = 930254 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.28 avg-zval = 33.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930254 bytes = 3372192 } seq expand size = 6511778 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4446 value = 3.4956 dict = 1.00 all = 3.6096 } Zip/UnZip{ index = 0.0957 value = 0.2861 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 9.536924 GB avg = 0.165 KB (by entry num) total key len = 1.596825 GB avg = 0.028 KB total ukey len = 1.109309 GB avg = 0.016 KB total ukey num = 0.071121989 Billion total entry num = 0.057929718 Billion write speed all = 0.541053500 MB/sec (with seq num) write speed all = 0.518532368 MB/sec (without seq num) 2024/11/23-14:38:10.168643 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263849: 930254 keys, 37183214 bytes, temperature: kWarm 2024/11/23-14:38:10.168732 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343890168691, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263849, "file_size": 37183214, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 4012776165, "table_properties": {"data_size": 30944752, "index_size": 1781312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26047112, "raw_average_key_size": 28, "raw_value_size": 108170664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930254, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343886, "oldest_key_time": 0, "file_creation_time": 1732343886, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263849, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:10.204641 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263850.sst) /dev/shm/ToplingTemp/Topling-77035-QKRs9c 2024/11/23-14:38:11.317469 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263850.sst): first pass time = 1.11's, 120.614'MB/sec 2024/11/23-14:38:13.161592 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263850.sst): old prealloc_size = 147639500, real_size = 32318912 2024/11/23-14:38:13.162158 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263850.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934763 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0303 GB avg-val = 115.58 avg-zval = 32.42 2024/11/23-14:38:13.162950 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263850.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:14.023019 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263850.sst): second pass time = 1.84's, 58.614'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.305'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3246.045'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.318'MB/sec write SST data time = 0.13's, 290.394'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35702.259'MB/sec zip my value time = 1.84's, unzip length = 0.108'GB zip my value throughput = 58.614'MB/sec zip pipeline throughput = 130.023'MB/sec entries = 934763 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.58 avg-zval = 32.42 usrkeys = 934763 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.58 avg-zval = 32.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934763 bytes = 3271680 } seq expand size = 6543341 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5141 value = 3.5650 dict = 1.00 all = 3.6832 } Zip/UnZip{ index = 0.0951 value = 0.2805 dict = 1.00 all = 0.2715 } ---------------------------- total value len = 9.651512 GB avg = 0.164 KB (by entry num) total key len = 1.622998 GB avg = 0.028 KB total ukey len = 1.128004 GB avg = 0.016 KB total ukey num = 0.072056752 Billion total entry num = 0.058864481 Billion write speed all = 0.547790041 MB/sec (with seq num) write speed all = 0.524909843 MB/sec (without seq num) 2024/11/23-14:38:14.069296 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263850: 934763 keys, 36440826 bytes, temperature: kWarm 2024/11/23-14:38:14.069365 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343894069329, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263850, "file_size": 36440826, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776176, "table_properties": {"data_size": 30306928, "index_size": 1778112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26173364, "raw_average_key_size": 28, "raw_value_size": 108044426, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934763, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343890, "oldest_key_time": 0, "file_creation_time": 1732343890, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263850, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:14.106369 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263851.sst) /dev/shm/ToplingTemp/Topling-77035-AmZHsd 2024/11/23-14:38:15.073520 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263851.sst): first pass time = 0.97's, 138.782'MB/sec 2024/11/23-14:38:16.381472 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263851.sst): old prealloc_size = 147639500, real_size = 32161824 2024/11/23-14:38:16.382063 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263851.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931885 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0302 GB avg-val = 116.03 avg-zval = 32.36 2024/11/23-14:38:16.382854 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263851.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:17.442255 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263851.sst): second pass time = 1.31's, 82.710'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 34.196'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3118.971'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.264'MB/sec write SST data time = 0.18's, 203.987'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34846.261'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.710'MB/sec zip pipeline throughput = 82.710'MB/sec entries = 931885 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.03 avg-zval = 32.36 usrkeys = 931885 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.03 avg-zval = 32.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931885 bytes = 3261616 } seq expand size = 6523195 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5143 value = 3.5855 dict = 1.00 all = 3.7001 } Zip/UnZip{ index = 0.0951 value = 0.2789 dict = 1.00 all = 0.2703 } ---------------------------- total value len = 9.766160 GB avg = 0.163 KB (by entry num) total key len = 1.649091 GB avg = 0.028 KB total ukey len = 1.146642 GB avg = 0.016 KB total ukey num = 0.072988637 Billion total entry num = 0.059796366 Billion write speed all = 0.554536045 MB/sec (with seq num) write speed all = 0.531297490 MB/sec (without seq num) 2024/11/23-14:38:17.487170 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263851: 931885 keys, 36274000 bytes, temperature: kWarm 2024/11/23-14:38:17.487231 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343897487198, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263851, "file_size": 36274000, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 4012776192, "table_properties": {"data_size": 30156048, "index_size": 1772608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26092780, "raw_average_key_size": 28, "raw_value_size": 108125074, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931885, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343894, "oldest_key_time": 0, "file_creation_time": 1732343894, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263851, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:17.533703 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263852.sst) /dev/shm/ToplingTemp/Topling-77035-FSCUVe 2024/11/23-14:38:18.624132 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263852.sst): first pass time = 1.09's, 123.091'MB/sec 2024/11/23-14:38:19.997195 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263852.sst): old prealloc_size = 147639500, real_size = 34704640 2024/11/23-14:38:19.997883 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263852.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1085627 prefix = 4 raw-key = 0.0174 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1038 GB zip-val = 0.0324 GB avg-val = 95.63 avg-zval = 29.80 2024/11/23-14:38:19.998388 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263852.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:20.710489 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263852.sst): second pass time = 1.37's, 75.654'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 50.917'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3125.884'MB/sec (index lex order gen) rebuild zvType time = 0.57's, 0.475'MB/sec write SST data time = 0.14's, 257.302'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.036'MB, throughput = 33637.373'MB/sec zip my value time = 1.37's, unzip length = 0.104'GB zip my value throughput = 75.654'MB/sec zip pipeline throughput = 75.654'MB/sec entries = 1085627 avg-key = 28.00 avg-zkey = 1.91 avg-val = 95.63 avg-zval = 29.80 usrkeys = 1085627 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.96 avg-zval = 29.80 TagRS{ kinds = RS_Key0_TagN bytes = 169720 } TagArray{ size = 205498 bytes = 667888 } seq expand size = 1438486 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0217 GB value = 0.1038 GB dict = 1.04 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0324 GB dict = 1.04 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4468 value = 3.2088 dict = 1.00 all = 3.6957 } Zip/UnZip{ index = 0.0957 value = 0.3116 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 9.871419 GB avg = 0.162 KB (by entry num) total key len = 1.679488 GB avg = 0.028 KB total ukey len = 1.168355 GB avg = 0.016 KB total ukey num = 0.074074264 Billion total entry num = 0.060881993 Billion write speed all = 0.561036950 MB/sec (with seq num) write speed all = 0.537380247 MB/sec (without seq num) 2024/11/23-14:38:20.756177 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263852: 1085627 keys, 36317569 bytes, temperature: kWarm 2024/11/23-14:38:20.756274 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343900756210, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263852, "file_size": 36317569, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 32354672, "index_size": 2078400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30397556, "raw_average_key_size": 28, "raw_value_size": 103820187, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1085627, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343897, "oldest_key_time": 0, "file_creation_time": 1732343897, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263852, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:20.793016 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263853.sst) /dev/shm/ToplingTemp/Topling-77035-KRdAZf 2024/11/23-14:38:21.732220 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263853.sst): first pass time = 0.94's, 142.911'MB/sec 2024/11/23-14:38:23.129627 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263853.sst): old prealloc_size = 147639500, real_size = 35890656 2024/11/23-14:38:23.130247 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263853.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923424 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0339 GB avg-val = 117.35 avg-zval = 36.71 2024/11/23-14:38:23.130704 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263853.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:23.890354 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263853.sst): second pass time = 1.40's, 77.591'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 50.550'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3032.960'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.363'MB/sec write SST data time = 0.12's, 307.707'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34855.405'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.591'MB/sec zip pipeline throughput = 77.591'MB/sec entries = 923424 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.35 avg-zval = 36.71 usrkeys = 923424 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.50 avg-zval = 36.71 TagRS{ kinds = RS_Key0_TagN bytes = 144400 } TagArray{ size = 415150 bytes = 1556832 } seq expand size = 2906050 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5130 value = 3.1962 dict = 1.00 all = 3.4904 } Zip/UnZip{ index = 0.0951 value = 0.3129 dict = 1.00 all = 0.2865 } ---------------------------- total value len = 9.982687 GB avg = 0.162 KB (by entry num) total key len = 1.705344 GB avg = 0.028 KB total ukey len = 1.186823 GB avg = 0.016 KB total ukey num = 0.074997688 Billion total entry num = 0.061805417 Billion write speed all = 0.567609502 MB/sec (with seq num) write speed all = 0.543597696 MB/sec (without seq num) 2024/11/23-14:38:23.937274 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263853: 923424 keys, 38453521 bytes, temperature: kWarm 2024/11/23-14:38:23.937344 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343903937309, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263853, "file_size": 38453521, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3966839195, "table_properties": {"data_size": 33902864, "index_size": 1756736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25855872, "raw_average_key_size": 28, "raw_value_size": 108361956, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343900, "oldest_key_time": 0, "file_creation_time": 1732343900, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263853, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:23.975662 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263854.sst) /dev/shm/ToplingTemp/Topling-77035-Y1DfYf 2024/11/23-14:38:24.863968 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263854.sst): first pass time = 0.89's, 151.101'MB/sec 2024/11/23-14:38:26.159935 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263854.sst): old prealloc_size = 147639500, real_size = 32371328 2024/11/23-14:38:26.160568 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263854.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931429 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.10 avg-zval = 32.60 2024/11/23-14:38:26.161583 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263854.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:27.007880 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263854.sst): second pass time = 1.30's, 83.497'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.163'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2904.822'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 289.176'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35481.357'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.497'MB/sec zip pipeline throughput = 83.497'MB/sec entries = 931429 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 32.60 usrkeys = 931429 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 32.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931429 bytes = 3376448 } seq expand size = 6520003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5148 value = 3.5611 dict = 1.00 all = 3.6673 } Zip/UnZip{ index = 0.0951 value = 0.2808 dict = 1.00 all = 0.2727 } ---------------------------- total value len = 10.097344 GB avg = 0.161 KB (by entry num) total key len = 1.731424 GB avg = 0.028 KB total ukey len = 1.205452 GB avg = 0.016 KB total ukey num = 0.075929117 Billion total entry num = 0.062736846 Billion write speed all = 0.574357239 MB/sec (with seq num) write speed all = 0.549987256 MB/sec (without seq num) 2024/11/23-14:38:27.052648 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263854: 931429 keys, 36598956 bytes, temperature: kWarm 2024/11/23-14:38:27.052728 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343907052690, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263854, "file_size": 36598956, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775885, "table_properties": {"data_size": 30366640, "index_size": 1771648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080012, "raw_average_key_size": 28, "raw_value_size": 108137762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343903, "oldest_key_time": 0, "file_creation_time": 1732343903, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263854, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:27.094577 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263855.sst) /dev/shm/ToplingTemp/Topling-77035-sPqKze 2024/11/23-14:38:28.017076 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263855.sst): first pass time = 0.92's, 145.500'MB/sec 2024/11/23-14:38:29.318726 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263855.sst): old prealloc_size = 147639500, real_size = 32293008 2024/11/23-14:38:29.319310 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263855.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 116.12 avg-zval = 32.52 2024/11/23-14:38:29.320110 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263855.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:30.170092 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263855.sst): second pass time = 1.30's, 83.158'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.761'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3146.533'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 281.165'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33281.358'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.158'MB/sec zip pipeline throughput = 83.158'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 32.52 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 32.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5120 value = 3.5704 dict = 1.00 all = 3.6869 } Zip/UnZip{ index = 0.0951 value = 0.2801 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 10.212005 GB avg = 0.160 KB (by entry num) total key len = 1.757501 GB avg = 0.028 KB total ukey len = 1.224078 GB avg = 0.016 KB total ukey num = 0.076860426 Billion total entry num = 0.063668155 Billion write speed all = 0.581101633 MB/sec (with seq num) write speed all = 0.556373681 MB/sec (without seq num) 2024/11/23-14:38:30.216424 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263855: 931309 keys, 36403900 bytes, temperature: kWarm 2024/11/23-14:38:30.216500 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343910216459, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263855, "file_size": 36403900, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775902, "table_properties": {"data_size": 30288064, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343907, "oldest_key_time": 0, "file_creation_time": 1732343907, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263855, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:30.256761 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263856.sst) /dev/shm/ToplingTemp/Topling-77035-BYdN4e 2024/11/23-14:38:31.281340 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263856.sst): first pass time = 1.02's, 131.003'MB/sec 2024/11/23-14:38:32.580827 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263856.sst): old prealloc_size = 147639500, real_size = 32506928 2024/11/23-14:38:32.581405 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263856.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 116.09 avg-zval = 32.75 2024/11/23-14:38:32.582295 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263856.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:33.437460 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263856.sst): second pass time = 1.30's, 83.296'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.020'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3199.243'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 285.083'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 24212.540'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.296'MB/sec zip pipeline throughput = 83.296'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 32.75 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 32.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5122 value = 3.5453 dict = 1.00 all = 3.6422 } Zip/UnZip{ index = 0.0951 value = 0.2821 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 10.326662 GB avg = 0.160 KB (by entry num) total key len = 1.783582 GB avg = 0.028 KB total ukey len = 1.242707 GB avg = 0.016 KB total ukey num = 0.077791888 Billion total entry num = 0.064599617 Billion write speed all = 0.587841007 MB/sec (with seq num) write speed all = 0.562755267 MB/sec (without seq num) 2024/11/23-14:38:33.482540 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263856: 931462 keys, 36851080 bytes, temperature: kWarm 2024/11/23-14:38:33.482611 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343913482574, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263856, "file_size": 36851080, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 30501728, "index_size": 1772160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343910, "oldest_key_time": 0, "file_creation_time": 1732343910, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263856, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:33.536647 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263857.sst) /dev/shm/ToplingTemp/Topling-77035-MJkwbd 2024/11/23-14:38:34.481776 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263857.sst): first pass time = 0.95's, 142.017'MB/sec 2024/11/23-14:38:35.811877 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263857.sst): old prealloc_size = 147639500, real_size = 33928176 2024/11/23-14:38:35.812429 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263857.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0319 GB avg-val = 117.04 avg-zval = 34.51 2024/11/23-14:38:35.813264 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263857.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:36.658167 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263857.sst): second pass time = 1.33's, 81.514'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.066'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3321.335'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 298.147'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34915.899'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.514'MB/sec zip pipeline throughput = 81.514'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 34.51 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 34.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3354608 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5137 value = 3.3913 dict = 1.00 all = 3.5193 } Zip/UnZip{ index = 0.0951 value = 0.2949 dict = 1.00 all = 0.2842 } ---------------------------- total value len = 10.441446 GB avg = 0.159 KB (by entry num) total key len = 1.809493 GB avg = 0.028 KB total ukey len = 1.261215 GB avg = 0.016 KB total ukey num = 0.078717293 Billion total entry num = 0.065525022 Billion write speed all = 0.594577531 MB/sec (with seq num) write speed all = 0.569136409 MB/sec (without seq num) 2024/11/23-14:38:36.706705 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263857: 925405 keys, 38137978 bytes, temperature: kWarm 2024/11/23-14:38:36.706776 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343916706741, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263857, "file_size": 38137978, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775947, "table_properties": {"data_size": 31936288, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343913, "oldest_key_time": 0, "file_creation_time": 1732343913, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263857, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:36.744457 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263858.sst) /dev/shm/ToplingTemp/Topling-77035-RQ7xMe 2024/11/23-14:38:37.639354 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263858.sst): first pass time = 0.89's, 149.988'MB/sec 2024/11/23-14:38:38.935140 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263858.sst): old prealloc_size = 147639500, real_size = 33247824 2024/11/23-14:38:38.935664 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263858.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923089 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0312 GB avg-val = 117.40 avg-zval = 33.85 2024/11/23-14:38:38.936498 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263858.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:39.783247 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263858.sst): second pass time = 1.29's, 83.686'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.161'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3467.250'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 295.563'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34107.475'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.686'MB/sec zip pipeline throughput = 83.686'MB/sec entries = 923089 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.40 avg-zval = 33.85 usrkeys = 923089 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.40 avg-zval = 33.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923089 bytes = 3346208 } seq expand size = 6461623 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4448 value = 3.4680 dict = 1.00 all = 3.5837 } Zip/UnZip{ index = 0.0957 value = 0.2884 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 10.556279 GB avg = 0.159 KB (by entry num) total key len = 1.835340 GB avg = 0.028 KB total ukey len = 1.279677 GB avg = 0.016 KB total ukey num = 0.079640382 Billion total entry num = 0.066448111 Billion write speed all = 0.601313952 MB/sec (with seq num) write speed all = 0.575518338 MB/sec (without seq num) 2024/11/23-14:38:39.828448 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263858: 923089 keys, 37451914 bytes, temperature: kWarm 2024/11/23-14:38:39.828521 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343919828482, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263858, "file_size": 37451914, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775987, "table_properties": {"data_size": 31249344, "index_size": 1767552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25846492, "raw_average_key_size": 28, "raw_value_size": 108371238, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923089, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343916, "oldest_key_time": 0, "file_creation_time": 1732343916, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263858, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:39.868357 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263859.sst) /dev/shm/ToplingTemp/Topling-77035-BQSFwd 2024/11/23-14:38:40.751597 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263859.sst): first pass time = 0.88's, 151.967'MB/sec 2024/11/23-14:38:42.076124 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263859.sst): old prealloc_size = 147639500, real_size = 33922112 2024/11/23-14:38:42.076661 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263859.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931782 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0319 GB avg-val = 116.04 avg-zval = 34.25 2024/11/23-14:38:42.078168 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263859.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:42.927911 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263859.sst): second pass time = 1.32's, 81.689'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.790'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3431.711'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 297.292'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36481.211'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.689'MB/sec zip pipeline throughput = 81.689'MB/sec entries = 931782 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.04 avg-zval = 34.25 usrkeys = 931782 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.04 avg-zval = 34.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931782 bytes = 3377728 } seq expand size = 6522474 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5135 value = 3.3878 dict = 1.00 all = 3.5181 } Zip/UnZip{ index = 0.0951 value = 0.2952 dict = 1.00 all = 0.2842 } ---------------------------- total value len = 10.670929 GB avg = 0.158 KB (by entry num) total key len = 1.861429 GB avg = 0.028 KB total ukey len = 1.298313 GB avg = 0.016 KB total ukey num = 0.080572164 Billion total entry num = 0.067379893 Billion write speed all = 0.608050711 MB/sec (with seq num) write speed all = 0.581897364 MB/sec (without seq num) 2024/11/23-14:38:42.974833 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263859: 931782 keys, 38150366 bytes, temperature: kWarm 2024/11/23-14:38:42.974913 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343922974869, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263859, "file_size": 38150366, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4012775229, "table_properties": {"data_size": 31916464, "index_size": 1772544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26089896, "raw_average_key_size": 28, "raw_value_size": 108127976, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931782, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343919, "oldest_key_time": 0, "file_creation_time": 1732343919, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263859, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:43.013946 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263860.sst) /dev/shm/ToplingTemp/Topling-77035-oQqS7b 2024/11/23-14:38:43.874525 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263860.sst): first pass time = 0.86's, 155.980'MB/sec 2024/11/23-14:38:45.218921 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263860.sst): old prealloc_size = 147639500, real_size = 35172896 2024/11/23-14:38:45.219487 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263860.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930745 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.20 avg-zval = 35.61 2024/11/23-14:38:45.220200 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263860.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:46.033805 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263860.sst): second pass time = 1.34's, 80.498'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 47.834'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3303.345'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.339'MB/sec write SST data time = 0.13's, 303.249'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36098.828'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.498'MB/sec zip pipeline throughput = 80.498'MB/sec entries = 930745 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.20 avg-zval = 35.61 usrkeys = 930745 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.83 avg-zval = 35.61 TagRS{ kinds = RS_Key0_TagN bytes = 145520 } TagArray{ size = 747748 bytes = 2804064 } seq expand size = 5234236 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.3793 value = 3.2630 dict = 1.00 all = 3.4434 } Zip/UnZip{ index = 0.0963 value = 0.3065 dict = 1.00 all = 0.2904 } ---------------------------- total value len = 10.784321 GB avg = 0.158 KB (by entry num) total key len = 1.887490 GB avg = 0.028 KB total ukey len = 1.316927 GB avg = 0.016 KB total ukey num = 0.081502909 Billion total entry num = 0.068310638 Billion write speed all = 0.614724077 MB/sec (with seq num) write speed all = 0.588213458 MB/sec (without seq num) 2024/11/23-14:38:46.082278 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263860: 930745 keys, 38977744 bytes, temperature: kWarm 2024/11/23-14:38:46.082351 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343926082313, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263860, "file_size": 38977744, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 33146576, "index_size": 1793472, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26060860, "raw_average_key_size": 28, "raw_value_size": 108156994, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930745, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343923, "oldest_key_time": 0, "file_creation_time": 1732343923, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263860, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:46.122013 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263861.sst) /dev/shm/ToplingTemp/Topling-77035-74I4hc 2024/11/23-14:38:47.040662 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263861.sst): first pass time = 0.92's, 146.111'MB/sec 2024/11/23-14:38:48.348691 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263861.sst): old prealloc_size = 147639500, real_size = 32775888 2024/11/23-14:38:48.349314 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.49 avg-zval = 33.12 2024/11/23-14:38:48.351072 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263861.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:49.201360 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263861.sst): second pass time = 1.31's, 82.783'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.526'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2968.025'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 286.081'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34803.843'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.783'MB/sec zip pipeline throughput = 82.783'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.49 avg-zval = 33.12 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.49 avg-zval = 33.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3483408 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4445 value = 3.5173 dict = 1.00 all = 3.6165 } Zip/UnZip{ index = 0.0957 value = 0.2843 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 10.899031 GB avg = 0.157 KB (by entry num) total key len = 1.913500 GB avg = 0.028 KB total ukey len = 1.335506 GB avg = 0.016 KB total ukey num = 0.082431814 Billion total entry num = 0.069239543 Billion write speed all = 0.621455079 MB/sec (with seq num) write speed all = 0.594588091 MB/sec (without seq num) 2024/11/23-14:38:49.247034 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263861: 928905 keys, 37112562 bytes, temperature: kWarm 2024/11/23-14:38:49.247104 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343929247068, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263861, "file_size": 37112562, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3288382583, "largest_seqno": 4012776210, "table_properties": {"data_size": 30764736, "index_size": 1778752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343926, "oldest_key_time": 0, "file_creation_time": 1732343926, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:49.289278 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263862.sst) /dev/shm/ToplingTemp/Topling-77035-yUqvFe 2024/11/23-14:38:50.317016 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263862.sst): first pass time = 1.03's, 130.601'MB/sec 2024/11/23-14:38:51.624945 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263862.sst): old prealloc_size = 147639500, real_size = 32552064 2024/11/23-14:38:51.625502 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932110 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.99 avg-zval = 32.77 2024/11/23-14:38:51.627063 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263862.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:52.485466 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263862.sst): second pass time = 1.31's, 82.741'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.745'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3483.454'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 286.294'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35945.524'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.741'MB/sec zip pipeline throughput = 82.741'MB/sec entries = 932110 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.99 avg-zval = 32.77 usrkeys = 932110 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.99 avg-zval = 32.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932110 bytes = 3378912 } seq expand size = 6524770 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5138 value = 3.5396 dict = 1.00 all = 3.6490 } Zip/UnZip{ index = 0.0951 value = 0.2825 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 11.013675 GB avg = 0.157 KB (by entry num) total key len = 1.939599 GB avg = 0.028 KB total ukey len = 1.354148 GB avg = 0.016 KB total ukey num = 0.083363924 Billion total entry num = 0.070171653 Billion write speed all = 0.628181548 MB/sec (with seq num) write speed all = 0.600957210 MB/sec (without seq num) 2024/11/23-14:38:52.530519 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263862: 932110 keys, 36781946 bytes, temperature: kWarm 2024/11/23-14:38:52.530591 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343932530554, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263862, "file_size": 36781946, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30545712, "index_size": 1773120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26099080, "raw_average_key_size": 28, "raw_value_size": 108118680, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932110, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343929, "oldest_key_time": 0, "file_creation_time": 1732343929, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:52.569143 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263863.sst) /dev/shm/ToplingTemp/Topling-77035-rd0ZPf 2024/11/23-14:38:53.556230 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263863.sst): first pass time = 0.99's, 135.981'MB/sec 2024/11/23-14:38:54.883817 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263863.sst): old prealloc_size = 147639500, real_size = 34007712 2024/11/23-14:38:54.884388 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263863.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933722 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0320 GB avg-val = 115.74 avg-zval = 34.26 2024/11/23-14:38:54.885989 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263863.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:55.730163 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263863.sst): second pass time = 1.33's, 81.489'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.325'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3273.555'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 300.149'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32438.886'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.489'MB/sec zip pipeline throughput = 81.489'MB/sec entries = 933722 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.74 avg-zval = 34.26 usrkeys = 933722 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.74 avg-zval = 34.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933722 bytes = 3501472 } seq expand size = 6536054 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4441 value = 3.3788 dict = 1.00 all = 3.4990 } Zip/UnZip{ index = 0.0957 value = 0.2960 dict = 1.00 all = 0.2858 } ---------------------------- total value len = 11.128285 GB avg = 0.157 KB (by entry num) total key len = 1.965743 GB avg = 0.028 KB total ukey len = 1.372822 GB avg = 0.016 KB total ukey num = 0.084297646 Billion total entry num = 0.071105375 Billion write speed all = 0.634907638 MB/sec (with seq num) write speed all = 0.607325386 MB/sec (without seq num) 2024/11/23-14:38:55.776457 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263863: 933722 keys, 38359398 bytes, temperature: kWarm 2024/11/23-14:38:55.776526 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343935776490, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263863, "file_size": 38359398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 31986064, "index_size": 1788032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26144216, "raw_average_key_size": 28, "raw_value_size": 108073608, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933722, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343932, "oldest_key_time": 0, "file_creation_time": 1732343932, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263863, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:55.828024 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263864.sst) /dev/shm/ToplingTemp/Topling-77035-qJFY2b 2024/11/23-14:38:56.781002 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263864.sst): first pass time = 0.95's, 140.847'MB/sec 2024/11/23-14:38:58.050526 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263864.sst): old prealloc_size = 147639500, real_size = 30557408 2024/11/23-14:38:58.051110 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263864.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928436 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0285 GB avg-val = 116.56 avg-zval = 30.74 2024/11/23-14:38:58.052699 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263864.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:38:58.875152 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263864.sst): second pass time = 1.27's, 85.326'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.312'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3204.182'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 269.398'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33785.573'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.326'MB/sec zip pipeline throughput = 85.326'MB/sec entries = 928436 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.56 avg-zval = 30.74 usrkeys = 928436 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.56 avg-zval = 30.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928436 bytes = 3481648 } seq expand size = 6499052 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.3757 value = 3.7925 dict = 1.00 all = 3.8468 } Zip/UnZip{ index = 0.0964 value = 0.2637 dict = 1.00 all = 0.2600 } ---------------------------- total value len = 11.243005 GB avg = 0.156 KB (by entry num) total key len = 1.991739 GB avg = 0.028 KB total ukey len = 1.391391 GB avg = 0.016 KB total ukey num = 0.085226082 Billion total entry num = 0.072033811 Billion write speed all = 0.641632920 MB/sec (with seq num) write speed all = 0.613694782 MB/sec (without seq num) 2024/11/23-14:38:58.918853 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263864: 928436 keys, 34890514 bytes, temperature: kWarm 2024/11/23-14:38:58.918925 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343938918888, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263864, "file_size": 34890514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3413799853, "largest_seqno": 4012776300, "table_properties": {"data_size": 28535504, "index_size": 1789632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25996208, "raw_average_key_size": 28, "raw_value_size": 108221620, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928436, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343935, "oldest_key_time": 0, "file_creation_time": 1732343935, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263864, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:38:58.955029 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263865.sst) /dev/shm/ToplingTemp/Topling-77035-VDWuff 2024/11/23-14:38:59.961890 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263865.sst): first pass time = 1.01's, 133.309'MB/sec 2024/11/23-14:39:01.318302 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263865.sst): old prealloc_size = 147639500, real_size = 35294608 2024/11/23-14:39:01.318847 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263865.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925504 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 117.02 avg-zval = 35.98 2024/11/23-14:39:01.320469 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263865.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:02.161685 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263865.sst): second pass time = 1.36's, 79.913'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.980'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3382.724'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 306.188'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34105.964'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.913'MB/sec zip pipeline throughput = 79.913'MB/sec entries = 925504 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.02 avg-zval = 35.98 usrkeys = 925504 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.02 avg-zval = 35.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925504 bytes = 3470656 } seq expand size = 6478528 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5133 value = 3.2521 dict = 1.00 all = 3.3876 } Zip/UnZip{ index = 0.0951 value = 0.3075 dict = 1.00 all = 0.2952 } ---------------------------- total value len = 11.357787 GB avg = 0.156 KB (by entry num) total key len = 2.017653 GB avg = 0.028 KB total ukey len = 1.409901 GB avg = 0.016 KB total ukey num = 0.086151586 Billion total entry num = 0.072959315 Billion write speed all = 0.648350710 MB/sec (with seq num) write speed all = 0.620058125 MB/sec (without seq num) 2024/11/23-14:39:02.209304 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263865: 925504 keys, 39620546 bytes, temperature: kWarm 2024/11/23-14:39:02.209375 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343942209340, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263865, "file_size": 39620546, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33302400, "index_size": 1760640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25914112, "raw_average_key_size": 28, "raw_value_size": 108303752, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925504, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343938, "oldest_key_time": 0, "file_creation_time": 1732343938, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263865, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:02.255149 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263866.sst) /dev/shm/ToplingTemp/Topling-77035-SgWffg 2024/11/23-14:39:03.221052 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263866.sst): first pass time = 0.97's, 138.963'MB/sec 2024/11/23-14:39:04.518015 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263866.sst): old prealloc_size = 147639500, real_size = 31210784 2024/11/23-14:39:04.518587 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263866.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0292 GB avg-val = 116.15 avg-zval = 31.37 2024/11/23-14:39:04.520211 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263866.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:05.355642 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263866.sst): second pass time = 1.30's, 83.457'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.446'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3237.746'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 273.833'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31318.563'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.457'MB/sec zip pipeline throughput = 83.457'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.15 avg-zval = 31.37 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 31.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5132 value = 3.7029 dict = 1.00 all = 3.7750 } Zip/UnZip{ index = 0.0951 value = 0.2701 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 11.472453 GB avg = 0.155 KB (by entry num) total key len = 2.043724 GB avg = 0.028 KB total ukey len = 1.428523 GB avg = 0.016 KB total ukey num = 0.087082669 Billion total entry num = 0.073890398 Billion write speed all = 0.655071190 MB/sec (with seq num) write speed all = 0.626421980 MB/sec (without seq num) 2024/11/23-14:39:05.399036 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263866: 931083 keys, 35553988 bytes, temperature: kWarm 2024/11/23-14:39:05.399106 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343945399070, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263866, "file_size": 35553988, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 29206544, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343942, "oldest_key_time": 0, "file_creation_time": 1732343942, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263866, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:05.437963 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263867.sst) /dev/shm/ToplingTemp/Topling-77035-FVo5Pf 2024/11/23-14:39:06.303991 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263867.sst): first pass time = 0.87's, 154.994'MB/sec 2024/11/23-14:39:07.579004 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263867.sst): old prealloc_size = 147639500, real_size = 31189152 2024/11/23-14:39:07.579559 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263867.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925040 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0292 GB avg-val = 117.09 avg-zval = 31.54 2024/11/23-14:39:07.581615 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263867.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:08.399351 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263867.sst): second pass time = 1.27's, 85.039'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.238'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3357.955'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 270.878'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34371.649'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.039'MB/sec zip pipeline throughput = 85.039'MB/sec entries = 925040 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.09 avg-zval = 31.54 usrkeys = 925040 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.93 avg-zval = 31.54 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 903875 bytes = 3276560 } seq expand size = 6327125 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3752 value = 3.7127 dict = 1.00 all = 3.7840 } Zip/UnZip{ index = 0.0964 value = 0.2693 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 11.587096 GB avg = 0.155 KB (by entry num) total key len = 2.069625 GB avg = 0.028 KB total ukey len = 1.447023 GB avg = 0.016 KB total ukey num = 0.088007709 Billion total entry num = 0.074815438 Billion write speed all = 0.661785174 MB/sec (with seq num) write speed all = 0.632781581 MB/sec (without seq num) 2024/11/23-14:39:08.442469 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263867: 925040 keys, 35469642 bytes, temperature: kWarm 2024/11/23-14:39:08.442537 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343948442503, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263867, "file_size": 35469642, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 29174544, "index_size": 1783168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25901120, "raw_average_key_size": 28, "raw_value_size": 108316624, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925040, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343945, "oldest_key_time": 0, "file_creation_time": 1732343945, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263867, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:08.479449 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263868.sst) /dev/shm/ToplingTemp/Topling-77035-QdLEGd 2024/11/23-14:39:09.271523 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263868.sst): first pass time = 0.79's, 169.459'MB/sec 2024/11/23-14:39:10.783197 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263868.sst): old prealloc_size = 147639500, real_size = 36651328 2024/11/23-14:39:10.783744 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263868.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926941 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0346 GB avg-val = 116.80 avg-zval = 37.38 2024/11/23-14:39:10.783886 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263868.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:11.493718 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263868.sst): second pass time = 1.51's, 71.657'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.080'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3395.933'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.397'MB/sec write SST data time = 0.13's, 296.991'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35067.273'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.657'MB/sec zip pipeline throughput = 71.657'MB/sec entries = 926941 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.80 avg-zval = 37.38 usrkeys = 926941 avg-key = 20.00 avg-zkey = 1.91 avg-val = 116.96 avg-zval = 37.38 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 21118 bytes = 58096 } seq expand size = 147826 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4445 value = 3.1250 dict = 1.00 all = 3.5589 } Zip/UnZip{ index = 0.0957 value = 0.3200 dict = 1.00 all = 0.2810 } ---------------------------- total value len = 11.695508 GB avg = 0.154 KB (by entry num) total key len = 2.095579 GB avg = 0.028 KB total ukey len = 1.465562 GB avg = 0.016 KB total ukey num = 0.088934650 Billion total entry num = 0.075742379 Billion write speed all = 0.668196143 MB/sec (with seq num) write speed all = 0.638837606 MB/sec (without seq num) 2024/11/23-14:39:11.595581 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263868: 926941 keys, 37713175 bytes, temperature: kWarm 2024/11/23-14:39:11.595634 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343951595604, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263868, "file_size": 37713175, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3254334261, "table_properties": {"data_size": 34644464, "index_size": 1774976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954348, "raw_average_key_size": 28, "raw_value_size": 108263410, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926941, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343948, "oldest_key_time": 0, "file_creation_time": 1732343948, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263868, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:11.643581 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263869.sst) /dev/shm/ToplingTemp/Topling-77035-6z24Re 2024/11/23-14:39:12.577077 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263869.sst): first pass time = 0.93's, 143.787'MB/sec 2024/11/23-14:39:14.662878 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263869.sst): old prealloc_size = 147639500, real_size = 37934368 2024/11/23-14:39:14.663461 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263869.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925030 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0359 GB avg-val = 117.10 avg-zval = 38.84 2024/11/23-14:39:14.664066 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263869.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:39:15.793708 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263869.sst): second pass time = 2.09's, 51.947'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.13's, 36.260'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3075.173'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.261'MB/sec write SST data time = 0.24's, 167.937'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33141.114'MB/sec zip my value time = 2.09's, unzip length = 0.108'GB zip my value throughput = 51.947'MB/sec zip pipeline throughput = 95.045'MB/sec entries = 925030 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.10 avg-zval = 38.84 usrkeys = 925030 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.26 avg-zval = 38.84 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 550425 bytes = 2064112 } seq expand size = 3852975 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0359 GB dict = 1.08 MB all = 0.0410 GB } UnZip/Zip{ index = 10.4460 value = 3.0145 dict = 1.00 all = 3.2734 } Zip/UnZip{ index = 0.0957 value = 0.3317 dict = 1.00 all = 0.3055 } ---------------------------- total value len = 11.807678 GB avg = 0.154 KB (by entry num) total key len = 2.121480 GB avg = 0.028 KB total ukey len = 1.484063 GB avg = 0.017 KB total ukey num = 0.089859680 Billion total entry num = 0.076667409 Billion write speed all = 0.674745279 MB/sec (with seq num) write speed all = 0.645034381 MB/sec (without seq num) 2024/11/23-14:39:15.848566 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263869: 925030 keys, 41002094 bytes, temperature: kWarm 2024/11/23-14:39:15.848655 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343955848607, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263869, "file_size": 41002094, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4030479689, "table_properties": {"data_size": 35931856, "index_size": 1771072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900840, "raw_average_key_size": 28, "raw_value_size": 108316992, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925030, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343951, "oldest_key_time": 0, "file_creation_time": 1732343951, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263869, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:15.908354 139957248734784 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263870.sst) /dev/shm/ToplingTemp/Topling-77035-zGvfVf 2024/11/23-14:39:16.065717 139957248734784 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263870.sst): first pass time = 0.16's, 113.341'MB/sec 2024/11/23-14:39:17.001826 139957248734784 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263870.sst): old prealloc_size = 147639500, real_size = 5431008 2024/11/23-14:39:17.002007 139957248734784 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263870.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 122914 prefix = 4 raw-key = 0.0020 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0144 GB zip-val = 0.0052 GB avg-val = 117.06 avg-zval = 42.02 2024/11/23-14:39:17.002171 139957248734784 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263870.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/23-14:39:17.199907 139957248734784 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00/263870.sst): second pass time = 0.94's, 15.385'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 29.973'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1625.029'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.173'MB/sec write SST data time = 0.02's, 290.738'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.151'MB, throughput = 4081.637'MB/sec zip my value time = 0.94's, unzip length = 0.014'GB zip my value throughput = 15.385'MB/sec zip pipeline throughput = 106.390'MB/sec entries = 122914 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.06 avg-zval = 42.02 usrkeys = 122914 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.06 avg-zval = 42.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 122914 bytes = 384128 } seq expand size = 860398 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0025 GB value = 0.0144 GB dict = 0.15 MB all = 0.0178 GB } __ZipSize{ index = 0.0002 GB value = 0.0052 GB dict = 0.15 MB all = 0.0059 GB } UnZip/Zip{ index = 10.4433 value = 2.7860 dict = 1.00 all = 3.0033 } Zip/UnZip{ index = 0.0958 value = 0.3589 dict = 1.00 all = 0.3330 } ---------------------------- total value len = 11.822927 GB avg = 0.154 KB (by entry num) total key len = 2.124922 GB avg = 0.028 KB total ukey len = 1.486521 GB avg = 0.017 KB total ukey num = 0.089982594 Billion total entry num = 0.076790323 Billion write speed all = 0.675604652 MB/sec (with seq num) write speed all = 0.645848148 MB/sec (without seq num) 2024/11/23-14:39:17.213358 139957248734784 [compaction_job.cc:1948] [default] [JOB 28087] Generated table #263870: 122914 keys, 5936783 bytes, temperature: kWarm 2024/11/23-14:39:17.213452 139957248734784 EVENT_LOG_v1 {"time_micros": 1732343957213399, "cf_name": "default", "job": 28087, "event": "table_file_creation", "file_number": 263870, "file_size": 5936783, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012776798, "largest_seqno": 4030479922, "table_properties": {"data_size": 5164688, "index_size": 235392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3441592, "raw_average_key_size": 28, "raw_value_size": 14388620, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 122914, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343955, "oldest_key_time": 0, "file_creation_time": 1732343955, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263870, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:39:17.229124 139957248734784 [compaction_job.cc:703] job-28087: subcompact[0], size: 0.982810 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28087/att-00 [263844,263845,263846,263847,263848,263849,263850,263851,263852,263853,263854,263855,263856,263857,263858,263859,263860,263861,263862,263863,263864,263865,263866,263867,263868,263869,263870] 2024/11/23-14:39:17.456712 139957248734784 (Original Log Time 2024/11/23-14:37:46.835966) EVENT_LOG_v1 {"time_micros": 1732343866835937, "job": 28087, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [263823], "files_L3": [263726, 263782, 263784, 263785, 263786, 263787, 263788, 263789, 263790, 263791, 263792, 263793, 263794, 263795, 263796, 263797, 263798, 263799, 263800, 263801, 263802, 263803, 263804, 263805, 263806, 263838], "score": 0.998312, "input_data_size": 1008185529, "oldest_snapshot_seqno": 4030928832} 2024/11/23-14:39:17.456719 139957248734784 (Original Log Time 2024/11/23-14:39:17.454203) [compaction_job.cc:2022] [default] [JOB 28087] Compacted 1@2 + 26@3 files to L3 => 982809664 bytes 2024/11/23-14:39:17.456722 139957248734784 (Original Log Time 2024/11/23-14:39:17.456585) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 27 0 0 0] max score 0.04, MB/sec: 11.2 rd, 10.9 wr, level 3, files in(1, 26) out(27 +0 blob) MB in(27.2, 934.3 +0.0 blob) out(937.3 +0.0 blob), read-write-amplify(69.9) write-amplify(34.5) OK, records in: 24543205, records dropped: 111046 output_compression: Snappy 2024/11/23-14:39:17.456730 139957248734784 (Original Log Time 2024/11/23-14:39:17.456658) EVENT_LOG_v1 {"time_micros": 1732343957456623, "cf": "default", "job": 28087, "event": "compaction_finished", "compaction_time_micros": 90393178, "compaction_time_cpu_micros": 38802690, "output_level": 3, "num_output_files": 27, "total_output_size": 982809664, "num_input_records": 24543205, "num_output_records": 24432159, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 711535941, "file_range_sync_nanos": 0, "file_fsync_nanos": 18808, "file_prepare_write_nanos": 84028, "lsm_state": [0, 0, 0, 27, 0, 0, 0]} 2024/11/23-14:39:17.577278 139957248734784 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732343866,"executesMs":90760,"compactionJobId":28087,"attempt":0,"compactionInputRawBytes":3523261587,"compactionInputZipBytes":1008185529,"compactionOutputRawBytes":0,"compactionOutputZipBytes":982809664,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-a168f7398046f225632b305cb6ddf6fe-f0934fc071911a69-00"}}, response = 2024/11/23-14:39:17.577368 139957248734784 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28087/att-00: olev 3, work 90.760 s, result 23.686 ms, install 2.567 ms, input{raw 3.281 GiB zip 961.481 MiB}