2024/11/23-18:33:25.295530 139957508908608 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00: fork to child time = 0.021838 sec 2024/11/23-18:33:25.296554 139957508908608 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00 2024/11/23-18:33:25.304727 139957508908608 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00 2024/11/23-18:33:25.305930 139957508908608 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-18:33:25.306282 139957508908608 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-18:33:25.359261 139957508908608 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33215/att-00/MANIFEST-206347 2024/11/23-18:33:25.397280 139957508908608 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33215/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 275075, last_sequence is 4186300787, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 274901 2024/11/23-18:33:25.397290 139957508908608 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-18:33:25.397442 139957508908608 [version_set.cc:5721] Creating manifest 275075 2024/11/23-18:33:25.432591 139957508908608 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-18:33:25.453716 139957508908608 [compaction_job.cc:2437] [default] [JOB 33215] Compacting 1@2 + 20@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-18:33:25.453730 139957508908608 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24870 Base level 2, inputs: [275035(27MB)], [275072(37MB) 275073(3669KB) 274982(39MB) 274983(37MB) 274984(36MB) 274985(36MB) 274986(38MB) 274987(35MB) 274988(34MB) 274989(37MB) 274990(34MB) 274991(37MB) 274992(35MB) 274993(35MB) 274994(36MB) 274995(36MB) 275003(36MB) 275004(36MB) 275005(37MB) 275064(35MB)] 2024/11/23-18:33:25.462275 139957508908608 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:33:25.462298 139957508908608 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:33:25.462313 139957508908608 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-18:33:25.462317 139957508908608 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:33:25.462327 139957508908608 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-18:33:26.083419 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275076.sst) /dev/shm/ToplingTemp/Topling-151511-FuH7QL 2024/11/23-18:33:26.732615 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275076.sst): first pass time = 0.65's, 206.765'MB/sec 2024/11/23-18:33:28.340931 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275076.sst): old prealloc_size = 147639500, real_size = 35098288 2024/11/23-18:33:28.341514 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275076.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928248 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.59 avg-zval = 35.65 2024/11/23-18:33:28.342280 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275076.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:33:29.369296 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275076.sst): second pass time = 1.61's, 67.332'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 37.894'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3347.850'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.257'MB/sec write SST data time = 0.13's, 309.543'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 38841.333'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.332'MB/sec zip pipeline throughput = 67.332'MB/sec entries = 928248 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.59 avg-zval = 35.65 usrkeys = 928248 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.59 avg-zval = 35.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928248 bytes = 3016816 } seq expand size = 6497736 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4461 value = 3.2708 dict = 1.00 all = 3.4443 } Zip/UnZip{ index = 0.0957 value = 0.3057 dict = 1.00 all = 0.2903 } ---------------------------- total value len = 8.963344 GB avg = 0.168 KB (by entry num) total key len = 1.466812 GB avg = 0.028 KB total ukey len = 1.016443 GB avg = 0.015 KB total ukey num = 0.066478683 Billion total entry num = 0.053286412 Billion write speed all = 0.300605525 MB/sec (with seq num) write speed all = 0.288319465 MB/sec (without seq num) 2024/11/23-18:33:29.418364 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275076: 928248 keys, 38968468 bytes, temperature: kWarm 2024/11/23-18:33:29.418454 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358009418399, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275076, "file_size": 38968468, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4131604997, "largest_seqno": 4186079864, "table_properties": {"data_size": 33088800, "index_size": 1777216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25990944, "raw_average_key_size": 28, "raw_value_size": 108226820, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928248, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358006, "oldest_key_time": 0, "file_creation_time": 1732358006, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275076, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:33:29.455343 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275077.sst) /dev/shm/ToplingTemp/Topling-151511-ixpT1L 2024/11/23-18:33:30.551638 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275077.sst): first pass time = 1.10's, 122.433'MB/sec 2024/11/23-18:33:32.100958 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275077.sst): old prealloc_size = 147639500, real_size = 36535376 2024/11/23-18:33:32.101676 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275077.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934013 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0345 GB avg-val = 115.70 avg-zval = 36.95 2024/11/23-18:33:32.102602 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275077.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:33:33.131175 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275077.sst): second pass time = 1.55's, 69.790'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 39.693'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2621.905'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.267'MB/sec write SST data time = 0.15's, 264.017'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32365.367'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.790'MB/sec zip pipeline throughput = 69.790'MB/sec entries = 934013 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.70 avg-zval = 36.95 usrkeys = 934013 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.70 avg-zval = 36.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934013 bytes = 3502560 } seq expand size = 6538091 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0409 GB } UnZip/Zip{ index = 10.4448 value = 3.1311 dict = 1.00 all = 3.2822 } Zip/UnZip{ index = 0.0957 value = 0.3194 dict = 1.00 all = 0.3047 } ---------------------------- total value len = 9.077948 GB avg = 0.167 KB (by entry num) total key len = 1.492964 GB avg = 0.028 KB total ukey len = 1.035123 GB avg = 0.015 KB total ukey num = 0.067412696 Billion total entry num = 0.054220425 Billion write speed all = 0.304629195 MB/sec (with seq num) write speed all = 0.292129139 MB/sec (without seq num) 2024/11/23-18:33:33.182269 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275077: 934013 keys, 40892026 bytes, temperature: kWarm 2024/11/23-18:33:33.182339 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358013182303, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275077, "file_size": 40892026, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4186080104, "table_properties": {"data_size": 34513216, "index_size": 1788480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26152364, "raw_average_key_size": 28, "raw_value_size": 108065450, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934013, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358009, "oldest_key_time": 0, "file_creation_time": 1732358009, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275077, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:33:33.222157 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275078.sst) /dev/shm/ToplingTemp/Topling-151511-rj7lnM 2024/11/23-18:33:34.282161 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275078.sst): first pass time = 1.06's, 126.625'MB/sec 2024/11/23-18:33:35.750382 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275078.sst): old prealloc_size = 147639500, real_size = 36869728 2024/11/23-18:33:35.751100 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275078.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927783 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0348 GB avg-val = 116.67 avg-zval = 37.56 2024/11/23-18:33:35.751821 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275078.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:33:36.635114 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275078.sst): second pass time = 1.47's, 73.796'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 45.440'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2562.707'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.15's, 262.496'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25632.221'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.796'MB/sec zip pipeline throughput = 73.796'MB/sec entries = 927783 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.67 avg-zval = 37.56 usrkeys = 927783 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.35 avg-zval = 37.56 TagRS{ kinds = RS_Key0_TagN bytes = 145080 } TagArray{ size = 620991 bytes = 2328736 } seq expand size = 4346937 multi value expand size = 0 cnt WriteAppend = 21 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.3762 value = 3.1059 dict = 1.00 all = 3.3387 } Zip/UnZip{ index = 0.0964 value = 0.3220 dict = 1.00 all = 0.2995 } ---------------------------- total value len = 9.190535 GB avg = 0.167 KB (by entry num) total key len = 1.518942 GB avg = 0.028 KB total ukey len = 1.053679 GB avg = 0.015 KB total ukey num = 0.068340479 Billion total entry num = 0.055148208 Billion write speed all = 0.308591151 MB/sec (with seq num) write speed all = 0.295878486 MB/sec (without seq num) 2024/11/23-18:33:36.684995 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275078: 927783 keys, 40201092 bytes, temperature: kWarm 2024/11/23-18:33:36.685064 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358016685029, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275078, "file_size": 40201092, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 34849296, "index_size": 1788288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25977924, "raw_average_key_size": 28, "raw_value_size": 108239918, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927783, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358013, "oldest_key_time": 0, "file_creation_time": 1732358013, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275078, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:33:36.722104 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275079.sst) /dev/shm/ToplingTemp/Topling-151511-HoexRI 2024/11/23-18:33:37.889167 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275079.sst): first pass time = 1.17's, 115.008'MB/sec 2024/11/23-18:33:39.307603 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275079.sst): old prealloc_size = 147639500, real_size = 34454256 2024/11/23-18:33:39.308351 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275079.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926357 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 116.89 avg-zval = 35.03 2024/11/23-18:33:39.309244 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275079.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:33:40.336402 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275079.sst): second pass time = 1.42's, 76.427'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 37.586'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2443.879'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.267'MB/sec write SST data time = 0.16's, 241.037'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35077.912'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.427'MB/sec zip pipeline throughput = 76.427'MB/sec entries = 926357 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.89 avg-zval = 35.03 usrkeys = 926357 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.89 avg-zval = 35.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926357 bytes = 3358064 } seq expand size = 6484499 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.4451 value = 3.3369 dict = 1.00 all = 3.4711 } Zip/UnZip{ index = 0.0957 value = 0.2997 dict = 1.00 all = 0.2881 } ---------------------------- total value len = 9.305299 GB avg = 0.166 KB (by entry num) total key len = 1.544880 GB avg = 0.028 KB total ukey len = 1.072206 GB avg = 0.015 KB total ukey num = 0.069266836 Billion total entry num = 0.056074565 Billion write speed all = 0.312612115 MB/sec (with seq num) write speed all = 0.299687286 MB/sec (without seq num) 2024/11/23-18:33:40.385405 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275079: 926357 keys, 38667180 bytes, temperature: kWarm 2024/11/23-18:33:40.385476 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358020385438, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275079, "file_size": 38667180, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775967, "table_properties": {"data_size": 32448736, "index_size": 1773760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25937996, "raw_average_key_size": 28, "raw_value_size": 108279786, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926357, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358016, "oldest_key_time": 0, "file_creation_time": 1732358016, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275079, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:33:40.422154 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275080.sst) /dev/shm/ToplingTemp/Topling-151511-8HRGUK 2024/11/23-18:33:41.671006 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275080.sst): first pass time = 1.25's, 107.476'MB/sec 2024/11/23-18:33:43.209556 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275080.sst): old prealloc_size = 147639500, real_size = 35055984 2024/11/23-18:33:43.210266 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275080.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925453 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0331 GB avg-val = 117.03 avg-zval = 35.73 2024/11/23-18:33:43.211133 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275080.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:33:44.085007 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275080.sst): second pass time = 1.54's, 70.428'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 44.854'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2547.634'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.16's, 250.678'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34616.390'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.428'MB/sec zip pipeline throughput = 70.428'MB/sec entries = 925453 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.03 avg-zval = 35.73 usrkeys = 925453 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.03 avg-zval = 35.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925453 bytes = 3354784 } seq expand size = 6478171 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5138 value = 3.2756 dict = 1.00 all = 3.4182 } Zip/UnZip{ index = 0.0951 value = 0.3053 dict = 1.00 all = 0.2926 } ---------------------------- total value len = 9.420082 GB avg = 0.165 KB (by entry num) total key len = 1.570793 GB avg = 0.028 KB total ukey len = 1.090715 GB avg = 0.016 KB total ukey num = 0.070192289 Billion total entry num = 0.057000018 Billion write speed all = 0.316631609 MB/sec (with seq num) write speed all = 0.303494889 MB/sec (without seq num) 2024/11/23-18:33:44.133468 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275080: 925453 keys, 39265748 bytes, temperature: kWarm 2024/11/23-18:33:44.133562 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358024133511, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275080, "file_size": 39265748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012776021, "table_properties": {"data_size": 33063968, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25912684, "raw_average_key_size": 28, "raw_value_size": 108305174, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925453, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358020, "oldest_key_time": 0, "file_creation_time": 1732358020, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275080, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:33:44.186106 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275081.sst) /dev/shm/ToplingTemp/Topling-151511-Su8nfM 2024/11/23-18:33:45.355396 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275081.sst): first pass time = 1.17's, 114.791'MB/sec 2024/11/23-18:33:46.965230 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275081.sst): old prealloc_size = 147639500, real_size = 36128912 2024/11/23-18:33:46.965914 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275081.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928491 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0341 GB avg-val = 116.55 avg-zval = 36.75 2024/11/23-18:33:46.966732 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275081.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:33:47.779314 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275081.sst): second pass time = 1.61's, 67.260'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 49.045'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2737.416'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.343'MB/sec write SST data time = 0.14's, 291.561'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33337.534'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.260'MB/sec zip pipeline throughput = 67.260'MB/sec entries = 928491 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.55 avg-zval = 36.75 usrkeys = 928491 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.18 avg-zval = 36.75 TagRS{ kinds = RS_Key0_TagN bytes = 145160 } TagArray{ size = 745494 bytes = 2795616 } seq expand size = 5218458 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0341 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.4462 value = 3.1718 dict = 1.00 all = 3.3616 } Zip/UnZip{ index = 0.0957 value = 0.3153 dict = 1.00 all = 0.2975 } ---------------------------- total value len = 9.533521 GB avg = 0.165 KB (by entry num) total key len = 1.596791 GB avg = 0.028 KB total ukey len = 1.109285 GB avg = 0.016 KB total ukey num = 0.071120780 Billion total entry num = 0.057928509 Billion write speed all = 0.320614447 MB/sec (with seq num) write speed all = 0.307265159 MB/sec (without seq num) 2024/11/23-18:33:47.831234 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275081: 928491 keys, 39926492 bytes, temperature: kWarm 2024/11/23-18:33:47.831291 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358027831260, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275081, "file_size": 39926492, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 34118912, "index_size": 1777664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997748, "raw_average_key_size": 28, "raw_value_size": 108219998, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928491, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358024, "oldest_key_time": 0, "file_creation_time": 1732358024, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275081, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:33:47.871986 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275082.sst) /dev/shm/ToplingTemp/Topling-151511-egHX3L 2024/11/23-18:33:49.097175 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275082.sst): first pass time = 1.23's, 109.552'MB/sec 2024/11/23-18:33:50.647505 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275082.sst): old prealloc_size = 147639500, real_size = 33126800 2024/11/23-18:33:50.648081 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275082.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.49 avg-zval = 33.50 2024/11/23-18:33:50.648865 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275082.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:33:51.521158 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275082.sst): second pass time = 1.55's, 69.828'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.750'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3192.102'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.312'MB/sec write SST data time = 0.13's, 291.754'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31767.991'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.828'MB/sec zip pipeline throughput = 69.828'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.49 avg-zval = 33.50 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.49 avg-zval = 33.50 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3367296 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4463 value = 3.4776 dict = 1.00 all = 3.5938 } Zip/UnZip{ index = 0.0957 value = 0.2876 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 9.648232 GB avg = 0.164 KB (by entry num) total key len = 1.622800 GB avg = 0.028 KB total ukey len = 1.127863 GB avg = 0.016 KB total ukey num = 0.072049685 Billion total entry num = 0.058857414 Billion write speed all = 0.324632974 MB/sec (with seq num) write speed all = 0.311071087 MB/sec (without seq num) 2024/11/23-18:33:51.566489 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275082: 928905 keys, 37347522 bytes, temperature: kWarm 2024/11/23-18:33:51.566563 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358031566525, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275082, "file_size": 37347522, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776210, "table_properties": {"data_size": 31115968, "index_size": 1778432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358027, "oldest_key_time": 0, "file_creation_time": 1732358027, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275082, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:33:51.614937 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275083.sst) /dev/shm/ToplingTemp/Topling-151511-DfRthJ 2024/11/23-18:33:52.619844 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275083.sst): first pass time = 1.00's, 133.571'MB/sec 2024/11/23-18:33:54.418088 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275083.sst): old prealloc_size = 147639500, real_size = 32154896 2024/11/23-18:33:54.418658 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275083.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936424 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0301 GB avg-val = 115.33 avg-zval = 32.19 2024/11/23-18:33:54.419504 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275083.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:33:55.408427 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275083.sst): second pass time = 1.80's, 60.111'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 36.755'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3225.784'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.310'MB/sec write SST data time = 0.23's, 155.245'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34731.906'MB/sec zip my value time = 1.80's, unzip length = 0.108'GB zip my value throughput = 60.111'MB/sec zip pipeline throughput = 109.575'MB/sec entries = 936424 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.33 avg-zval = 32.19 usrkeys = 936424 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.33 avg-zval = 32.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936424 bytes = 3394544 } seq expand size = 6554968 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5124 value = 3.5833 dict = 1.00 all = 3.6875 } Zip/UnZip{ index = 0.0951 value = 0.2791 dict = 1.00 all = 0.2712 } ---------------------------- total value len = 9.762784 GB avg = 0.163 KB (by entry num) total key len = 1.649020 GB avg = 0.028 KB total ukey len = 1.146591 GB avg = 0.016 KB total ukey num = 0.072986109 Billion total entry num = 0.059793838 Billion write speed all = 0.328650772 MB/sec (with seq num) write speed all = 0.314874657 MB/sec (without seq num) 2024/11/23-18:33:55.455655 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275083: 936424 keys, 36397996 bytes, temperature: kWarm 2024/11/23-18:33:55.455744 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358035455697, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275083, "file_size": 36397996, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30139008, "index_size": 1781568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26219872, "raw_average_key_size": 28, "raw_value_size": 107997868, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358031, "oldest_key_time": 0, "file_creation_time": 1732358031, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275083, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:33:55.534155 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275084.sst) /dev/shm/ToplingTemp/Topling-151511-ItnSCJ 2024/11/23-18:33:56.616552 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275084.sst): first pass time = 1.08's, 124.009'MB/sec 2024/11/23-18:33:57.975091 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275084.sst): old prealloc_size = 147639500, real_size = 35138016 2024/11/23-18:33:57.975658 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275084.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929408 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.41 avg-zval = 35.64 2024/11/23-18:33:57.977887 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275084.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:33:58.819903 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275084.sst): second pass time = 1.36's, 79.716'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.729'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3284.101'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 300.256'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33754.450'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.716'MB/sec zip pipeline throughput = 79.716'MB/sec entries = 929408 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.41 avg-zval = 35.64 usrkeys = 929408 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.41 avg-zval = 35.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929408 bytes = 3485296 } seq expand size = 6505856 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4433 value = 3.2662 dict = 1.00 all = 3.4000 } Zip/UnZip{ index = 0.0958 value = 0.3062 dict = 1.00 all = 0.2941 } ---------------------------- total value len = 9.877485 GB avg = 0.163 KB (by entry num) total key len = 1.675043 GB avg = 0.028 KB total ukey len = 1.165180 GB avg = 0.016 KB total ukey num = 0.073915517 Billion total entry num = 0.060723246 Billion write speed all = 0.332670817 MB/sec (with seq num) write speed all = 0.318681947 MB/sec (without seq num) 2024/11/23-18:33:58.868785 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275084: 929408 keys, 39475848 bytes, temperature: kWarm 2024/11/23-18:33:58.868859 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358038868821, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275084, "file_size": 39475848, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33125584, "index_size": 1779904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26023424, "raw_average_key_size": 28, "raw_value_size": 108194420, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929408, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358035, "oldest_key_time": 0, "file_creation_time": 1732358035, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275084, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:33:58.907442 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275085.sst) /dev/shm/ToplingTemp/Topling-151511-3AE6DL 2024/11/23-18:33:59.916154 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275085.sst): first pass time = 1.01's, 133.065'MB/sec 2024/11/23-18:34:01.200685 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275085.sst): old prealloc_size = 147639500, real_size = 30991456 2024/11/23-18:34:01.201300 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275085.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0290 GB avg-val = 116.13 avg-zval = 31.13 2024/11/23-18:34:01.202096 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275085.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:34:02.033596 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275085.sst): second pass time = 1.28's, 84.242'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.284'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3076.593'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 273.498'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33700.394'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.242'MB/sec zip pipeline throughput = 84.242'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 31.13 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 31.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.5120 value = 3.7308 dict = 1.00 all = 3.8110 } Zip/UnZip{ index = 0.0951 value = 0.2680 dict = 1.00 all = 0.2624 } ---------------------------- total value len = 9.992146 GB avg = 0.162 KB (by entry num) total key len = 1.701118 GB avg = 0.028 KB total ukey len = 1.183804 GB avg = 0.016 KB total ukey num = 0.074846759 Billion total entry num = 0.061654488 Billion write speed all = 0.336692357 MB/sec (with seq num) write speed all = 0.322490270 MB/sec (without seq num) 2024/11/23-18:34:02.078317 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275085: 931242 keys, 35218720 bytes, temperature: kWarm 2024/11/23-18:34:02.078387 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358042078350, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275085, "file_size": 35218720, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 28986704, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358038, "oldest_key_time": 0, "file_creation_time": 1732358038, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275085, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:34:02.115757 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275086.sst) /dev/shm/ToplingTemp/Topling-151511-LGVMQM 2024/11/23-18:34:03.029715 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275086.sst): first pass time = 0.91's, 146.862'MB/sec 2024/11/23-18:34:04.368253 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275086.sst): old prealloc_size = 147639500, real_size = 35253008 2024/11/23-18:34:04.368805 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275086.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.46 avg-zval = 36.05 2024/11/23-18:34:04.370385 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275086.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:34:05.212916 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275086.sst): second pass time = 1.34's, 81.021'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.850'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3311.631'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 306.205'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31916.370'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.021'MB/sec zip pipeline throughput = 81.021'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 36.05 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 36.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5135 value = 3.2580 dict = 1.00 all = 3.3918 } Zip/UnZip{ index = 0.0951 value = 0.3069 dict = 1.00 all = 0.2948 } ---------------------------- total value len = 10.106988 GB avg = 0.162 KB (by entry num) total key len = 1.726954 GB avg = 0.028 KB total ukey len = 1.202258 GB avg = 0.016 KB total ukey num = 0.075769456 Billion total entry num = 0.062577185 Billion write speed all = 0.340711768 MB/sec (with seq num) write speed all = 0.326298458 MB/sec (without seq num) 2024/11/23-18:34:05.261277 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275086: 922697 keys, 39571354 bytes, temperature: kWarm 2024/11/23-18:34:05.261347 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358045261311, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275086, "file_size": 39571354, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33266880, "index_size": 1755264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358042, "oldest_key_time": 0, "file_creation_time": 1732358042, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275086, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:34:05.306501 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275087.sst) /dev/shm/ToplingTemp/Topling-151511-eBZi0I 2024/11/23-18:34:06.306083 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275087.sst): first pass time = 1.00's, 134.279'MB/sec 2024/11/23-18:34:07.630794 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275087.sst): old prealloc_size = 147639500, real_size = 32464784 2024/11/23-18:34:07.631412 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275087.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.15 avg-zval = 32.70 2024/11/23-18:34:07.632983 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275087.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:34:08.478029 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275087.sst): second pass time = 1.32's, 81.687'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.447'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2995.144'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 288.696'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32527.598'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.687'MB/sec zip pipeline throughput = 81.687'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 32.70 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 32.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4441 value = 3.5518 dict = 1.00 all = 3.6464 } Zip/UnZip{ index = 0.0957 value = 0.2815 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 10.221653 GB avg = 0.161 KB (by entry num) total key len = 1.753024 GB avg = 0.028 KB total ukey len = 1.220880 GB avg = 0.016 KB total ukey num = 0.076700539 Billion total entry num = 0.063508268 Billion write speed all = 0.344731285 MB/sec (with seq num) write speed all = 0.330104894 MB/sec (without seq num) 2024/11/23-18:34:08.527168 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275087: 931083 keys, 36807972 bytes, temperature: kWarm 2024/11/23-18:34:08.527248 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358048527210, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275087, "file_size": 36807972, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30448832, "index_size": 1782976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358045, "oldest_key_time": 0, "file_creation_time": 1732358045, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275087, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:34:08.564686 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275088.sst) /dev/shm/ToplingTemp/Topling-151511-dsePFL 2024/11/23-18:34:09.390624 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275088.sst): first pass time = 0.83's, 162.511'MB/sec 2024/11/23-18:34:10.707146 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275088.sst): old prealloc_size = 147639500, real_size = 32713120 2024/11/23-18:34:10.707722 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275088.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926394 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 116.88 avg-zval = 33.15 2024/11/23-18:34:10.709279 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275088.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:34:11.525281 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275088.sst): second pass time = 1.32's, 82.320'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.540'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3269.805'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 281.011'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34747.435'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.320'MB/sec zip pipeline throughput = 82.320'MB/sec entries = 926394 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.88 avg-zval = 33.15 usrkeys = 926394 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.55 avg-zval = 33.15 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 750724 bytes = 2721392 } seq expand size = 5255068 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4433 value = 3.5262 dict = 1.00 all = 3.6834 } Zip/UnZip{ index = 0.0958 value = 0.2836 dict = 1.00 all = 0.2715 } ---------------------------- total value len = 10.335187 GB avg = 0.160 KB (by entry num) total key len = 1.778963 GB avg = 0.028 KB total ukey len = 1.239408 GB avg = 0.016 KB total ukey num = 0.077626933 Billion total entry num = 0.064434662 Billion write speed all = 0.348715888 MB/sec (with seq num) write speed all = 0.333877444 MB/sec (without seq num) 2024/11/23-18:34:11.570147 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275088: 926394 keys, 36438626 bytes, temperature: kWarm 2024/11/23-18:34:11.570233 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358051570190, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275088, "file_size": 36438626, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 30707216, "index_size": 1774144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939032, "raw_average_key_size": 28, "raw_value_size": 108278760, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926394, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358048, "oldest_key_time": 0, "file_creation_time": 1732358048, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275088, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:34:11.609515 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275089.sst) /dev/shm/ToplingTemp/Topling-151511-xHkZIJ 2024/11/23-18:34:12.611584 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275089.sst): first pass time = 1.00's, 133.946'MB/sec 2024/11/23-18:34:14.120482 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275089.sst): old prealloc_size = 147639500, real_size = 36814944 2024/11/23-18:34:14.121030 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275089.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926755 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.83 avg-zval = 37.56 2024/11/23-18:34:14.121273 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275089.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:34:14.831710 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275089.sst): second pass time = 1.51's, 71.793'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.853'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3363.002'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.394'MB/sec write SST data time = 0.12's, 312.752'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35470.453'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.793'MB/sec zip pipeline throughput = 71.793'MB/sec entries = 926755 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.83 avg-zval = 37.56 usrkeys = 926755 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.99 avg-zval = 37.56 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4447 value = 3.1104 dict = 1.00 all = 3.5043 } Zip/UnZip{ index = 0.0957 value = 0.3215 dict = 1.00 all = 0.2854 } ---------------------------- total value len = 10.444536 GB avg = 0.160 KB (by entry num) total key len = 1.804912 GB avg = 0.028 KB total ukey len = 1.257943 GB avg = 0.016 KB total ukey num = 0.078553688 Billion total entry num = 0.065361417 Billion write speed all = 0.352577013 MB/sec (with seq num) write speed all = 0.337526583 MB/sec (without seq num) 2024/11/23-18:34:14.879241 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275089: 926755 keys, 38300605 bytes, temperature: kWarm 2024/11/23-18:34:14.879310 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358054879274, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275089, "file_size": 38300605, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34808464, "index_size": 1774592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949140, "raw_average_key_size": 28, "raw_value_size": 108268678, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926755, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358051, "oldest_key_time": 0, "file_creation_time": 1732358051, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275089, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:34:14.919577 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275090.sst) /dev/shm/ToplingTemp/Topling-151511-AUIXQL 2024/11/23-18:34:15.844256 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275090.sst): first pass time = 0.92's, 145.156'MB/sec 2024/11/23-18:34:17.209964 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275090.sst): old prealloc_size = 147639500, real_size = 35797216 2024/11/23-18:34:17.210533 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275090.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925570 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0338 GB avg-val = 117.01 avg-zval = 36.52 2024/11/23-18:34:17.211084 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275090.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:34:17.993570 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275090.sst): second pass time = 1.36's, 79.367'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 49.520'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3404.367'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.353'MB/sec write SST data time = 0.13's, 303.795'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 38210.664'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.367'MB/sec zip pipeline throughput = 79.367'MB/sec entries = 925570 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.01 avg-zval = 36.52 usrkeys = 925570 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.34 avg-zval = 36.52 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 572004 bytes = 2002032 } seq expand size = 4004028 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5140 value = 3.2037 dict = 1.00 all = 3.4590 } Zip/UnZip{ index = 0.0951 value = 0.3121 dict = 1.00 all = 0.2891 } ---------------------------- total value len = 10.556842 GB avg = 0.159 KB (by entry num) total key len = 1.830828 GB avg = 0.028 KB total ukey len = 1.276454 GB avg = 0.016 KB total ukey num = 0.079479258 Billion total entry num = 0.066286987 Billion write speed all = 0.356523017 MB/sec (with seq num) write speed all = 0.341260850 MB/sec (without seq num) 2024/11/23-18:34:18.041611 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275090: 925570 keys, 38802802 bytes, temperature: kWarm 2024/11/23-18:34:18.041679 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358058041644, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275090, "file_size": 38802802, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4186080173, "table_properties": {"data_size": 33805008, "index_size": 1760640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915960, "raw_average_key_size": 28, "raw_value_size": 108301800, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925570, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358054, "oldest_key_time": 0, "file_creation_time": 1732358054, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275090, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:34:18.079218 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275091.sst) /dev/shm/ToplingTemp/Topling-151511-DUaVDK 2024/11/23-18:34:19.005753 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275091.sst): first pass time = 0.93's, 144.866'MB/sec 2024/11/23-18:34:20.341849 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275091.sst): old prealloc_size = 147639500, real_size = 34794608 2024/11/23-18:34:20.342432 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275091.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925643 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0328 GB avg-val = 117.00 avg-zval = 35.42 2024/11/23-18:34:20.343313 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275091.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:34:21.182456 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275091.sst): second pass time = 1.33's, 81.130'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.127'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3161.351'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 307.352'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33317.645'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.130'MB/sec zip pipeline throughput = 81.130'MB/sec entries = 925643 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.00 avg-zval = 35.42 usrkeys = 925643 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.00 avg-zval = 35.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925643 bytes = 3124064 } seq expand size = 6479501 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.4442 value = 3.3028 dict = 1.00 all = 3.4616 } Zip/UnZip{ index = 0.0957 value = 0.3028 dict = 1.00 all = 0.2889 } ---------------------------- total value len = 10.671621 GB avg = 0.159 KB (by entry num) total key len = 1.856746 GB avg = 0.028 KB total ukey len = 1.294967 GB avg = 0.016 KB total ukey num = 0.080404901 Billion total entry num = 0.067212630 Billion write speed all = 0.360539263 MB/sec (with seq num) write speed all = 0.345065392 MB/sec (without seq num) 2024/11/23-18:34:21.231949 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275091: 925643 keys, 38773394 bytes, temperature: kWarm 2024/11/23-18:34:21.232031 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358061231993, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275091, "file_size": 38773394, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4186079734, "table_properties": {"data_size": 32790496, "index_size": 1772544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25918004, "raw_average_key_size": 28, "raw_value_size": 108299790, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925643, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358058, "oldest_key_time": 0, "file_creation_time": 1732358058, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275091, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:34:21.267017 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275092.sst) /dev/shm/ToplingTemp/Topling-151511-ibAvJL 2024/11/23-18:34:22.313957 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275092.sst): first pass time = 1.05's, 128.204'MB/sec 2024/11/23-18:34:23.642687 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275092.sst): old prealloc_size = 147639500, real_size = 35258096 2024/11/23-18:34:23.643302 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275092.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929021 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0333 GB avg-val = 116.47 avg-zval = 35.80 2024/11/23-18:34:23.644063 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275092.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:34:24.473033 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275092.sst): second pass time = 1.33's, 81.481'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.127'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2978.718'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 310.494'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32633.657'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.481'MB/sec zip pipeline throughput = 81.481'MB/sec entries = 929021 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.47 avg-zval = 35.80 usrkeys = 929021 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.47 avg-zval = 35.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929021 bytes = 3019328 } seq expand size = 6503147 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5131 value = 3.2535 dict = 1.00 all = 3.4300 } Zip/UnZip{ index = 0.0951 value = 0.3074 dict = 1.00 all = 0.2915 } ---------------------------- total value len = 10.786329 GB avg = 0.158 KB (by entry num) total key len = 1.882759 GB avg = 0.028 KB total ukey len = 1.313548 GB avg = 0.016 KB total ukey num = 0.081333922 Billion total entry num = 0.068141651 Billion write speed all = 0.364554385 MB/sec (with seq num) write speed all = 0.348868118 MB/sec (without seq num) 2024/11/23-18:34:24.521297 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275092: 929021 keys, 39130578 bytes, temperature: kWarm 2024/11/23-18:34:24.521359 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358064521324, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275092, "file_size": 39130578, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4138852857, "largest_seqno": 4186080270, "table_properties": {"data_size": 33258272, "index_size": 1767360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26012588, "raw_average_key_size": 28, "raw_value_size": 108205214, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929021, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358061, "oldest_key_time": 0, "file_creation_time": 1732358061, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275092, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:34:24.557620 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275093.sst) /dev/shm/ToplingTemp/Topling-151511-3BF22I 2024/11/23-18:34:25.601294 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275093.sst): first pass time = 1.04's, 128.605'MB/sec 2024/11/23-18:34:26.937123 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275093.sst): old prealloc_size = 147639500, real_size = 34433312 2024/11/23-18:34:26.937694 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275093.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928131 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.61 avg-zval = 34.95 2024/11/23-18:34:26.938572 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275093.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:34:27.782777 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275093.sst): second pass time = 1.34's, 81.065'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.432'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3222.820'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 302.937'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32437.613'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.065'MB/sec zip pipeline throughput = 81.065'MB/sec entries = 928131 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.61 avg-zval = 34.95 usrkeys = 928131 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.61 avg-zval = 34.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928131 bytes = 3132464 } seq expand size = 6496917 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5129 value = 3.3368 dict = 1.00 all = 3.4935 } Zip/UnZip{ index = 0.0951 value = 0.2997 dict = 1.00 all = 0.2862 } ---------------------------- total value len = 10.901057 GB avg = 0.158 KB (by entry num) total key len = 1.908746 GB avg = 0.028 KB total ukey len = 1.332110 GB avg = 0.016 KB total ukey num = 0.082262053 Billion total entry num = 0.069069782 Billion write speed all = 0.368568366 MB/sec (with seq num) write speed all = 0.352669956 MB/sec (without seq num) 2024/11/23-18:34:27.829209 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275093: 928131 keys, 38418930 bytes, temperature: kWarm 2024/11/23-18:34:27.829279 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358067829242, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275093, "file_size": 38418930, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4096159344, "largest_seqno": 4186080050, "table_properties": {"data_size": 32435408, "index_size": 1765696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25987668, "raw_average_key_size": 28, "raw_value_size": 108230178, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928131, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358064, "oldest_key_time": 0, "file_creation_time": 1732358064, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275093, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:34:27.874836 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275094.sst) /dev/shm/ToplingTemp/Topling-151511-s8kOBK 2024/11/23-18:34:28.912626 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275094.sst): first pass time = 1.04's, 129.335'MB/sec 2024/11/23-18:34:30.251639 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275094.sst): old prealloc_size = 147639500, real_size = 33243968 2024/11/23-18:34:30.252256 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275094.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 938384 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1079 GB zip-val = 0.0312 GB avg-val = 115.03 avg-zval = 33.25 2024/11/23-18:34:30.252999 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275094.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:34:31.099734 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275094.sst): second pass time = 1.34's, 80.691'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.800'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3033.073'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 293.281'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 23199.423'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.691'MB/sec zip pipeline throughput = 80.691'MB/sec entries = 938384 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.03 avg-zval = 33.25 usrkeys = 938384 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.03 avg-zval = 33.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 938384 bytes = 3049760 } seq expand size = 6568688 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.3774 value = 3.4596 dict = 1.00 all = 3.6133 } Zip/UnZip{ index = 0.0964 value = 0.2890 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 11.015568 GB avg = 0.157 KB (by entry num) total key len = 1.935021 GB avg = 0.028 KB total ukey len = 1.350878 GB avg = 0.016 KB total ukey num = 0.083200437 Billion total entry num = 0.070008166 Billion write speed all = 0.372583570 MB/sec (with seq num) write speed all = 0.356470702 MB/sec (without seq num) 2024/11/23-18:34:31.144955 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275094: 938384 keys, 37145774 bytes, temperature: kWarm 2024/11/23-18:34:31.145033 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358071144997, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275094, "file_size": 37145774, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4124223986, "largest_seqno": 4186080232, "table_properties": {"data_size": 31200688, "index_size": 1808512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26274752, "raw_average_key_size": 28, "raw_value_size": 107943088, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 938384, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358067, "oldest_key_time": 0, "file_creation_time": 1732358067, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275094, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:34:31.182090 139957508908608 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275095.sst) /dev/shm/ToplingTemp/Topling-151511-mmGfGJ 2024/11/23-18:34:31.318872 139957508908608 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275095.sst): first pass time = 0.14's, 243.582'MB/sec 2024/11/23-18:34:31.649945 139957508908608 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275095.sst): old prealloc_size = 147639500, real_size = 9581584 2024/11/23-18:34:31.650184 139957508908608 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275095.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 229442 prefix = 4 raw-key = 0.0037 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0269 GB zip-val = 0.0091 GB avg-val = 117.17 avg-zval = 39.60 2024/11/23-18:34:31.650409 139957508908608 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275095.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.033 GB 2024/11/23-18:34:31.852868 139957508908608 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00/275095.sst): second pass time = 0.33's, 81.397'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 51.990'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2269.762'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.337'MB/sec write SST data time = 0.03's, 324.690'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.277'MB, throughput = 10043.030'MB/sec zip my value time = 0.33's, unzip length = 0.027'GB zip my value throughput = 81.397'MB/sec zip pipeline throughput = 81.397'MB/sec entries = 229442 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.17 avg-zval = 39.60 usrkeys = 229442 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.17 avg-zval = 39.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 229442 bytes = 745696 } seq expand size = 1606094 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0046 GB value = 0.0269 GB dict = 0.28 MB all = 0.0333 GB } __ZipSize{ index = 0.0004 GB value = 0.0091 GB dict = 0.28 MB all = 0.0105 GB } UnZip/Zip{ index = 10.4825 value = 2.9587 dict = 1.00 all = 3.1575 } Zip/UnZip{ index = 0.0954 value = 0.3380 dict = 1.00 all = 0.3167 } ---------------------------- total value len = 11.044058 GB avg = 0.157 KB (by entry num) total key len = 1.941446 GB avg = 0.028 KB total ukey len = 1.355467 GB avg = 0.016 KB total ukey num = 0.083429879 Billion total entry num = 0.070237608 Billion write speed all = 0.373579950 MB/sec (with seq num) write speed all = 0.357414625 MB/sec (without seq num) 2024/11/23-18:34:31.872367 139957508908608 [compaction_job.cc:1948] [default] [JOB 33215] Generated table #275095: 229442 keys, 10549077 bytes, temperature: kWarm 2024/11/23-18:34:31.872424 139957508908608 EVENT_LOG_v1 {"time_micros": 1732358071872393, "cf_name": "default", "job": 33215, "event": "table_file_creation", "file_number": 275095, "file_size": 10549077, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4128616041, "largest_seqno": 4186079687, "table_properties": {"data_size": 9086272, "index_size": 437760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6424376, "raw_average_key_size": 28, "raw_value_size": 26883968, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 229442, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732358071, "oldest_key_time": 0, "file_creation_time": 1732358071, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 275095, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:34:31.888103 139957508908608 [compaction_job.cc:703] job-33215: subcompact[0], size: 0.740300 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-33215/att-00 [275076,275077,275078,275079,275080,275081,275082,275083,275084,275085,275086,275087,275088,275089,275090,275091,275092,275093,275094,275095] 2024/11/23-18:34:32.009509 139957508908608 (Original Log Time 2024/11/23-18:33:25.453767) EVENT_LOG_v1 {"time_micros": 1732358005453740, "job": 33215, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [275035], "files_L3": [275072, 275073, 274982, 274983, 274984, 274985, 274986, 274987, 274988, 274989, 274990, 274991, 274992, 274993, 274994, 274995, 275003, 275004, 275005, 275064], "score": 0.991406, "input_data_size": 762219868, "oldest_snapshot_seqno": 4186300782} 2024/11/23-18:34:32.009512 139957508908608 (Original Log Time 2024/11/23-18:34:32.007791) [compaction_job.cc:2022] [default] [JOB 33215] Compacted 1@2 + 20@3 files to L3 => 740300204 bytes 2024/11/23-18:34:32.009514 139957508908608 (Original Log Time 2024/11/23-18:34:32.009438) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.5 rd, 11.1 wr, level 3, files in(1, 20) out(20 +0 blob) MB in(27.1, 699.8 +0.0 blob) out(706.0 +0.0 blob), read-write-amplify(52.8) write-amplify(26.0) OK, records in: 17959616, records dropped: 80172 output_compression: Snappy 2024/11/23-18:34:32.009517 139957508908608 (Original Log Time 2024/11/23-18:34:32.009482) EVENT_LOG_v1 {"time_micros": 1732358072009462, "cf": "default", "job": 33215, "event": "compaction_finished", "compaction_time_micros": 66434343, "compaction_time_cpu_micros": 28318819, "output_level": 3, "num_output_files": 20, "total_output_size": 740300204, "num_input_records": 17959616, "num_output_records": 17879444, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 545143420, "file_range_sync_nanos": 0, "file_fsync_nanos": 14653, "file_prepare_write_nanos": 54270, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-18:34:32.143505 139957508908608 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732358005,"executesMs":66704,"compactionJobId":33215,"attempt":0,"compactionInputRawBytes":2594831032,"compactionInputZipBytes":762219868,"compactionOutputRawBytes":0,"compactionOutputZipBytes":740300204,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-99eb6fdbb4de4f9055643c7140ddcd8c-36468c8d12b1fa1e-00"}}, response = 2024/11/23-18:34:32.143604 139957508908608 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-33215/att-00: olev 3, work 66.704 s, result 7.818 ms, install 1.742 ms, input{raw 2.417 GiB zip 726.910 MiB}