2024/11/23-17:49:11.651440 139957433374272 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00: fork to child time = 0.007759 sec 2024/11/23-17:49:11.652181 139957433374272 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00 2024/11/23-17:49:11.659446 139957433374272 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00 2024/11/23-17:49:11.660327 139957433374272 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:49:11.660565 139957433374272 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:49:11.710475 139957433374272 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32342/att-00/MANIFEST-206347 2024/11/23-17:49:11.727717 139957433374272 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32342/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 273197, last_sequence is 4160378427, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 273062 2024/11/23-17:49:11.727724 139957433374272 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:49:11.727838 139957433374272 [version_set.cc:5721] Creating manifest 273197 2024/11/23-17:49:11.771662 139957433374272 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:49:11.793533 139957433374272 [compaction_job.cc:2437] [default] [JOB 32342] Compacting 1@2 + 20@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-17:49:11.793558 139957433374272 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24209 Base level 2, inputs: [273193(27MB)], [273148(37MB) 273149(38MB) 273150(39MB) 273151(35MB) 273153(35MB) 273154(35MB) 273155(36MB) 273156(36MB) 273157(38MB) 273158(35MB) 273159(34MB) 273160(37MB) 273161(34MB) 273162(37MB) 273163(35MB) 273164(35MB) 273165(36MB) 273166(37MB) 273167(12MB) 273116(36MB)] 2024/11/23-17:49:11.800912 139957433374272 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:49:11.800949 139957433374272 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:49:11.800968 139957433374272 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:49:11.800986 139957433374272 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:49:11.801001 139957433374272 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:49:12.405957 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273198.sst) /dev/shm/ToplingTemp/Topling-138887-rX86jd 2024/11/23-17:49:12.988754 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273198.sst): first pass time = 0.58's, 230.319'MB/sec 2024/11/23-17:49:14.971650 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273198.sst): old prealloc_size = 147639500, real_size = 35606944 2024/11/23-17:49:14.972235 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273198.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927162 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0336 GB avg-val = 116.76 avg-zval = 36.24 2024/11/23-17:49:14.973057 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273198.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:15.907868 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273198.sst): second pass time = 1.98's, 54.617'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 42.165'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3125.579'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 306.863'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 45648.471'MB/sec zip my value time = 1.98's, unzip length = 0.108'GB zip my value throughput = 54.617'MB/sec zip pipeline throughput = 121.025'MB/sec entries = 927162 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.76 avg-zval = 36.24 usrkeys = 927162 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.76 avg-zval = 36.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927162 bytes = 3013296 } seq expand size = 6490134 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4448 value = 3.2220 dict = 1.00 all = 3.4001 } Zip/UnZip{ index = 0.0957 value = 0.3104 dict = 1.00 all = 0.2941 } ---------------------------- total value len = 8.963367 GB avg = 0.168 KB (by entry num) total key len = 1.466782 GB avg = 0.028 KB total ukey len = 1.016421 GB avg = 0.015 KB total ukey num = 0.066477597 Billion total entry num = 0.053285326 Billion write speed all = 0.325497710 MB/sec (with seq num) write speed all = 0.312194543 MB/sec (without seq num) 2024/11/23-17:49:15.960635 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273198: 927162 keys, 39474728 bytes, temperature: kWarm 2024/11/23-17:49:15.960757 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355355960681, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273198, "file_size": 39474728, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4094024708, "largest_seqno": 4160358207, "table_properties": {"data_size": 33599632, "index_size": 1775360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960536, "raw_average_key_size": 28, "raw_value_size": 108257220, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927162, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355352, "oldest_key_time": 0, "file_creation_time": 1732355352, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273198, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:16.035572 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273199.sst) /dev/shm/ToplingTemp/Topling-138887-Xzqdoa 2024/11/23-17:49:16.951766 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273199.sst): first pass time = 0.92's, 146.504'MB/sec 2024/11/23-17:49:18.336725 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273199.sst): old prealloc_size = 147639500, real_size = 35870704 2024/11/23-17:49:18.337282 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273199.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0339 GB avg-val = 116.49 avg-zval = 36.45 2024/11/23-17:49:18.338036 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273199.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:19.167576 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273199.sst): second pass time = 1.38's, 78.195'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.975'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3330.877'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 306.898'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25765.352'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.195'MB/sec zip pipeline throughput = 79.726'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.49 avg-zval = 36.45 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.49 avg-zval = 36.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3135072 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0399 GB } UnZip/Zip{ index = 10.4482 value = 3.1957 dict = 1.00 all = 3.3673 } Zip/UnZip{ index = 0.0957 value = 0.3129 dict = 1.00 all = 0.2970 } ---------------------------- total value len = 9.078078 GB avg = 0.167 KB (by entry num) total key len = 1.492791 GB avg = 0.028 KB total ukey len = 1.034999 GB avg = 0.015 KB total ukey num = 0.067406502 Billion total entry num = 0.054214231 Billion write speed all = 0.329855664 MB/sec (with seq num) write speed all = 0.316321964 MB/sec (without seq num) 2024/11/23-17:49:19.218429 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273199: 928905 keys, 39859002 bytes, temperature: kWarm 2024/11/23-17:49:19.218497 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355359218462, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273199, "file_size": 39859002, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668904, "largest_seqno": 4160358459, "table_properties": {"data_size": 33860192, "index_size": 1778112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208514, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355356, "oldest_key_time": 0, "file_creation_time": 1732355356, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273199, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:19.257332 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273200.sst) /dev/shm/ToplingTemp/Topling-138887-Uop2oa 2024/11/23-17:49:20.247697 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273200.sst): first pass time = 0.99's, 135.532'MB/sec 2024/11/23-17:49:21.960082 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273200.sst): old prealloc_size = 147639500, real_size = 36740784 2024/11/23-17:49:21.960637 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273200.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933017 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0347 GB avg-val = 115.85 avg-zval = 37.23 2024/11/23-17:49:21.961470 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273200.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:23.007345 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273200.sst): second pass time = 1.71's, 63.161'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 39.241'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3332.233'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.260'MB/sec write SST data time = 0.15's, 273.247'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35366.078'MB/sec zip my value time = 1.71's, unzip length = 0.108'GB zip my value throughput = 63.161'MB/sec zip pipeline throughput = 63.161'MB/sec entries = 933017 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.85 avg-zval = 37.23 usrkeys = 933017 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.85 avg-zval = 37.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933017 bytes = 3498832 } seq expand size = 6531119 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0411 GB } UnZip/Zip{ index = 10.5141 value = 3.1122 dict = 1.00 all = 3.2661 } Zip/UnZip{ index = 0.0951 value = 0.3213 dict = 1.00 all = 0.3062 } ---------------------------- total value len = 9.192702 GB avg = 0.167 KB (by entry num) total key len = 1.518915 GB avg = 0.028 KB total ukey len = 1.053660 GB avg = 0.015 KB total ukey num = 0.068339519 Billion total entry num = 0.055147248 Billion write speed all = 0.334207579 MB/sec (with seq num) write speed all = 0.320442616 MB/sec (without seq num) 2024/11/23-17:49:23.060007 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273200: 933017 keys, 41094646 bytes, temperature: kWarm 2024/11/23-17:49:23.060076 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355363060041, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273200, "file_size": 41094646, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4160357851, "table_properties": {"data_size": 34732576, "index_size": 1774784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26124476, "raw_average_key_size": 28, "raw_value_size": 108093294, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933017, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355359, "oldest_key_time": 0, "file_creation_time": 1732355359, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273200, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:23.097968 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273201.sst) /dev/shm/ToplingTemp/Topling-138887-trFIcd 2024/11/23-17:49:24.214331 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273201.sst): first pass time = 1.12's, 120.233'MB/sec 2024/11/23-17:49:25.723813 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273201.sst): old prealloc_size = 147639500, real_size = 35512304 2024/11/23-17:49:25.724529 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273201.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924469 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0335 GB avg-val = 117.18 avg-zval = 36.24 2024/11/23-17:49:25.725212 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273201.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:26.625075 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273201.sst): second pass time = 1.51's, 71.813'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 43.088'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2569.210'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.16's, 238.413'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32635.543'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.813'MB/sec zip pipeline throughput = 71.813'MB/sec entries = 924469 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.18 avg-zval = 36.24 usrkeys = 924469 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.86 avg-zval = 36.24 TagRS{ kinds = RS_Key0_TagN bytes = 144560 } TagArray{ size = 617677 bytes = 2316304 } seq expand size = 4323739 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.3767 value = 3.2339 dict = 1.00 all = 3.4563 } Zip/UnZip{ index = 0.0964 value = 0.3092 dict = 1.00 all = 0.2893 } ---------------------------- total value len = 9.305359 GB avg = 0.166 KB (by entry num) total key len = 1.544801 GB avg = 0.028 KB total ukey len = 1.072149 GB avg = 0.015 KB total ukey num = 0.069263988 Billion total entry num = 0.056071717 Billion write speed all = 0.338491935 MB/sec (with seq num) write speed all = 0.324497801 MB/sec (without seq num) 2024/11/23-17:49:26.674283 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273201: 924469 keys, 38833012 bytes, temperature: kWarm 2024/11/23-17:49:26.674354 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355366674317, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273201, "file_size": 38833012, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 33499168, "index_size": 1781824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25885132, "raw_average_key_size": 28, "raw_value_size": 108332706, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924469, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355363, "oldest_key_time": 0, "file_creation_time": 1732355363, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273201, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:26.714204 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273202.sst) /dev/shm/ToplingTemp/Topling-138887-A1N00a 2024/11/23-17:49:27.995826 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273202.sst): first pass time = 1.28's, 104.728'MB/sec 2024/11/23-17:49:29.556545 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273202.sst): old prealloc_size = 147639500, real_size = 33958336 2024/11/23-17:49:29.557275 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273202.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931491 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0319 GB avg-val = 116.09 avg-zval = 34.28 2024/11/23-17:49:29.558138 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273202.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:30.423249 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273202.sst): second pass time = 1.55's, 69.684'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.925'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2535.264'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.14's, 264.160'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35457.362'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.684'MB/sec zip pipeline throughput = 69.684'MB/sec entries = 931491 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.09 avg-zval = 34.28 usrkeys = 931491 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.09 avg-zval = 34.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931491 bytes = 3260240 } seq expand size = 6520437 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.3761 value = 3.3867 dict = 1.00 all = 3.5256 } Zip/UnZip{ index = 0.0964 value = 0.2953 dict = 1.00 all = 0.2836 } ---------------------------- total value len = 9.420015 GB avg = 0.165 KB (by entry num) total key len = 1.570882 GB avg = 0.028 KB total ukey len = 1.090779 GB avg = 0.016 KB total ukey num = 0.070195479 Billion total entry num = 0.057003208 Billion write speed all = 0.342841918 MB/sec (with seq num) write speed all = 0.328616991 MB/sec (without seq num) 2024/11/23-17:49:30.470347 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273202: 931491 keys, 38069720 bytes, temperature: kWarm 2024/11/23-17:49:30.470422 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355370470386, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273202, "file_size": 38069720, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775902, "table_properties": {"data_size": 31929840, "index_size": 1795456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26081748, "raw_average_key_size": 28, "raw_value_size": 108136122, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931491, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355366, "oldest_key_time": 0, "file_creation_time": 1732355366, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273202, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:30.508117 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273203.sst) /dev/shm/ToplingTemp/Topling-138887-T2vDn9 2024/11/23-17:49:31.778685 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273203.sst): first pass time = 1.27's, 105.639'MB/sec 2024/11/23-17:49:33.296733 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273203.sst): old prealloc_size = 147639500, real_size = 32955536 2024/11/23-17:49:33.297318 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273203.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0310 GB avg-val = 116.09 avg-zval = 33.23 2024/11/23-17:49:33.298157 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273203.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:34.160301 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273203.sst): second pass time = 1.52's, 71.265'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.193'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3129.704'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.317'MB/sec write SST data time = 0.13's, 291.711'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33403.687'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.265'MB/sec zip pipeline throughput = 71.265'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 33.23 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 33.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5129 value = 3.4939 dict = 1.00 all = 3.5984 } Zip/UnZip{ index = 0.0951 value = 0.2862 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 9.534673 GB avg = 0.165 KB (by entry num) total key len = 1.596963 GB avg = 0.028 KB total ukey len = 1.109408 GB avg = 0.016 KB total ukey num = 0.071126941 Billion total entry num = 0.057934670 Billion write speed all = 0.347191523 MB/sec (with seq num) write speed all = 0.332735839 MB/sec (without seq num) 2024/11/23-17:49:34.212260 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273203: 931462 keys, 37299624 bytes, temperature: kWarm 2024/11/23-17:49:34.212327 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355374212292, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273203, "file_size": 37299624, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 30950464, "index_size": 1772032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355370, "oldest_key_time": 0, "file_creation_time": 1732355370, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273203, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:34.248408 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273204.sst) /dev/shm/ToplingTemp/Topling-138887-KJLJpa 2024/11/23-17:49:35.390662 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273204.sst): first pass time = 1.14's, 117.507'MB/sec 2024/11/23-17:49:37.034944 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273204.sst): old prealloc_size = 147639500, real_size = 34568064 2024/11/23-17:49:37.035531 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273204.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 117.04 avg-zval = 35.20 2024/11/23-17:49:37.036397 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273204.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:37.908378 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273204.sst): second pass time = 1.64's, 65.916'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 44.398'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3187.237'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.14's, 275.751'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31332.966'MB/sec zip my value time = 1.64's, unzip length = 0.108'GB zip my value throughput = 65.916'MB/sec zip pipeline throughput = 78.484'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 35.20 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 35.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3354608 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5133 value = 3.3247 dict = 1.00 all = 3.4612 } Zip/UnZip{ index = 0.0951 value = 0.3008 dict = 1.00 all = 0.2889 } ---------------------------- total value len = 9.649457 GB avg = 0.164 KB (by entry num) total key len = 1.622875 GB avg = 0.028 KB total ukey len = 1.127916 GB avg = 0.016 KB total ukey num = 0.072052346 Billion total entry num = 0.058860075 Billion write speed all = 0.351538671 MB/sec (with seq num) write speed all = 0.336853799 MB/sec (without seq num) 2024/11/23-17:49:37.956988 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273204: 925405 keys, 38777842 bytes, temperature: kWarm 2024/11/23-17:49:37.957059 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355377957025, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273204, "file_size": 38777842, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775947, "table_properties": {"data_size": 32576112, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355374, "oldest_key_time": 0, "file_creation_time": 1732355374, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273204, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:37.993566 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273205.sst) /dev/shm/ToplingTemp/Topling-138887-Rs4TO9 2024/11/23-17:49:39.054444 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273205.sst): first pass time = 1.06's, 126.521'MB/sec 2024/11/23-17:49:40.589710 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273205.sst): old prealloc_size = 147639500, real_size = 35420560 2024/11/23-17:49:40.590261 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273205.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925062 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0334 GB avg-val = 117.09 avg-zval = 36.12 2024/11/23-17:49:40.592124 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273205.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:41.442061 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273205.sst): second pass time = 1.53's, 70.585'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.495'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3341.264'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 304.195'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32810.189'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.585'MB/sec zip pipeline throughput = 70.585'MB/sec entries = 925062 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.09 avg-zval = 36.12 usrkeys = 925062 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.09 avg-zval = 36.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925062 bytes = 3353360 } seq expand size = 6475434 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4452 value = 3.2413 dict = 1.00 all = 3.3868 } Zip/UnZip{ index = 0.0957 value = 0.3085 dict = 1.00 all = 0.2953 } ---------------------------- total value len = 9.764248 GB avg = 0.163 KB (by entry num) total key len = 1.648776 GB avg = 0.028 KB total ukey len = 1.146417 GB avg = 0.016 KB total ukey num = 0.072977408 Billion total entry num = 0.059785137 Billion write speed all = 0.355887107 MB/sec (with seq num) write speed all = 0.340973087 MB/sec (without seq num) 2024/11/23-17:49:41.493435 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273205: 925062 keys, 39629198 bytes, temperature: kWarm 2024/11/23-17:49:41.493506 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355381493470, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273205, "file_size": 39629198, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012776021, "table_properties": {"data_size": 33417856, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25901736, "raw_average_key_size": 28, "raw_value_size": 108316060, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925062, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355377, "oldest_key_time": 0, "file_creation_time": 1732355377, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273205, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:41.533035 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273206.sst) /dev/shm/ToplingTemp/Topling-138887-EV1cp9 2024/11/23-17:49:42.691908 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273206.sst): first pass time = 1.16's, 115.821'MB/sec 2024/11/23-17:49:44.534043 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273206.sst): old prealloc_size = 147639500, real_size = 35770816 2024/11/23-17:49:44.534626 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273206.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929834 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0338 GB avg-val = 116.35 avg-zval = 36.31 2024/11/23-17:49:44.535409 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273206.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:45.417045 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273206.sst): second pass time = 1.84's, 58.751'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 44.819'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3158.842'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.19's, 212.380'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36230.201'MB/sec zip my value time = 1.84's, unzip length = 0.108'GB zip my value throughput = 58.751'MB/sec zip pipeline throughput = 123.343'MB/sec entries = 929834 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.35 avg-zval = 36.31 usrkeys = 929834 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.97 avg-zval = 36.31 TagRS{ kinds = RS_Key0_TagN bytes = 145400 } TagArray{ size = 746837 bytes = 2800656 } seq expand size = 5227859 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4470 value = 3.2046 dict = 1.00 all = 3.3917 } Zip/UnZip{ index = 0.0957 value = 0.3120 dict = 1.00 all = 0.2948 } ---------------------------- total value len = 9.877659 GB avg = 0.163 KB (by entry num) total key len = 1.674812 GB avg = 0.028 KB total ukey len = 1.165014 GB avg = 0.016 KB total ukey num = 0.073907242 Billion total entry num = 0.060714971 Billion write speed all = 0.360190733 MB/sec (with seq num) write speed all = 0.345046634 MB/sec (without seq num) 2024/11/23-17:49:45.470035 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273206: 929834 keys, 39572686 bytes, temperature: kWarm 2024/11/23-17:49:45.470128 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355385470078, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273206, "file_size": 39572686, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012774830, "table_properties": {"data_size": 33758064, "index_size": 1780096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26035352, "raw_average_key_size": 28, "raw_value_size": 108182484, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929834, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355381, "oldest_key_time": 0, "file_creation_time": 1732355381, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273206, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:45.543926 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273207.sst) /dev/shm/ToplingTemp/Topling-138887-TWzQg9 2024/11/23-17:49:46.659995 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273207.sst): first pass time = 1.12's, 120.267'MB/sec 2024/11/23-17:49:48.208664 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273207.sst): old prealloc_size = 147639500, real_size = 33087712 2024/11/23-17:49:48.209215 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273207.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.49 avg-zval = 33.46 2024/11/23-17:49:48.211037 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273207.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:49.297929 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273207.sst): second pass time = 1.55's, 69.940'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.09's, 34.252'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3371.473'MB/sec (index lex order gen) rebuild zvType time = 0.95's, 0.245'MB/sec write SST data time = 0.14's, 267.986'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 41952.283'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.940'MB/sec zip pipeline throughput = 69.940'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.49 avg-zval = 33.46 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.49 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3367296 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4452 value = 3.4820 dict = 1.00 all = 3.5975 } Zip/UnZip{ index = 0.0957 value = 0.2872 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 9.992369 GB avg = 0.162 KB (by entry num) total key len = 1.700821 GB avg = 0.028 KB total ukey len = 1.183592 GB avg = 0.016 KB total ukey num = 0.074836147 Billion total entry num = 0.061643876 Billion write speed all = 0.364534092 MB/sec (with seq num) write speed all = 0.349160157 MB/sec (without seq num) 2024/11/23-17:49:49.366325 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273207: 928905 keys, 37308266 bytes, temperature: kWarm 2024/11/23-17:49:49.366399 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355389366358, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273207, "file_size": 37308266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776210, "table_properties": {"data_size": 31076688, "index_size": 1778624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355385, "oldest_key_time": 0, "file_creation_time": 1732355385, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273207, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:49.409772 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273208.sst) /dev/shm/ToplingTemp/Topling-138887-dBmBsb 2024/11/23-17:49:50.332316 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273208.sst): first pass time = 0.92's, 145.493'MB/sec 2024/11/23-17:49:51.901795 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273208.sst): old prealloc_size = 147639500, real_size = 32241648 2024/11/23-17:49:51.902399 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273208.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935082 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0302 GB avg-val = 115.54 avg-zval = 32.33 2024/11/23-17:49:51.904187 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273208.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:52.993787 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273208.sst): second pass time = 1.57's, 68.865'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.09's, 33.409'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3053.095'MB/sec (index lex order gen) rebuild zvType time = 0.95's, 0.247'MB/sec write SST data time = 0.14's, 252.630'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30248.380'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.865'MB/sec zip pipeline throughput = 68.865'MB/sec entries = 935082 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.54 avg-zval = 32.33 usrkeys = 935082 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.54 avg-zval = 32.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935082 bytes = 3389680 } seq expand size = 6545574 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5128 value = 3.5739 dict = 1.00 all = 3.6791 } Zip/UnZip{ index = 0.0951 value = 0.2798 dict = 1.00 all = 0.2718 } ---------------------------- total value len = 10.106950 GB avg = 0.162 KB (by entry num) total key len = 1.727003 GB avg = 0.028 KB total ukey len = 1.202294 GB avg = 0.016 KB total ukey num = 0.075771229 Billion total entry num = 0.062578958 Billion write speed all = 0.368879875 MB/sec (with seq num) write speed all = 0.353274529 MB/sec (without seq num) 2024/11/23-17:49:53.041164 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273208: 935082 keys, 36481494 bytes, temperature: kWarm 2024/11/23-17:49:53.041234 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355393041199, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273208, "file_size": 36481494, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30228768, "index_size": 1778944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26182296, "raw_average_key_size": 28, "raw_value_size": 108035496, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935082, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355389, "oldest_key_time": 0, "file_creation_time": 1732355389, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273208, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:53.086885 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273209.sst) /dev/shm/ToplingTemp/Topling-138887-BEFHqb 2024/11/23-17:49:54.146110 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273209.sst): first pass time = 1.06's, 126.718'MB/sec 2024/11/23-17:49:55.599948 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273209.sst): old prealloc_size = 147639500, real_size = 35043616 2024/11/23-17:49:55.600676 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273209.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930750 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0330 GB avg-val = 116.20 avg-zval = 35.49 2024/11/23-17:49:55.602810 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273209.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:56.625622 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273209.sst): second pass time = 1.45's, 74.470'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 38.401'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2573.970'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.269'MB/sec write SST data time = 0.16's, 244.846'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33645.010'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.470'MB/sec zip pipeline throughput = 74.470'MB/sec entries = 930750 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.20 avg-zval = 35.49 usrkeys = 930750 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.20 avg-zval = 35.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930750 bytes = 3490320 } seq expand size = 6515250 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4434 value = 3.2747 dict = 1.00 all = 3.4078 } Zip/UnZip{ index = 0.0958 value = 0.3054 dict = 1.00 all = 0.2934 } ---------------------------- total value len = 10.221623 GB avg = 0.161 KB (by entry num) total key len = 1.753064 GB avg = 0.028 KB total ukey len = 1.220909 GB avg = 0.016 KB total ukey num = 0.076701979 Billion total entry num = 0.063509708 Billion write speed all = 0.373224457 MB/sec (with seq num) write speed all = 0.357388802 MB/sec (without seq num) 2024/11/23-17:49:56.675491 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273209: 930750 keys, 39385276 bytes, temperature: kWarm 2024/11/23-17:49:56.675561 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355396675525, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273209, "file_size": 39385276, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33028304, "index_size": 1782464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26061000, "raw_average_key_size": 28, "raw_value_size": 108156792, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930750, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355393, "oldest_key_time": 0, "file_creation_time": 1732355393, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273209, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:49:56.728438 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273210.sst) /dev/shm/ToplingTemp/Topling-138887-VQeOo9 2024/11/23-17:49:57.663530 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273210.sst): first pass time = 0.94's, 143.541'MB/sec 2024/11/23-17:49:59.155758 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273210.sst): old prealloc_size = 147639500, real_size = 30243696 2024/11/23-17:49:59.156349 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273210.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0282 GB avg-val = 116.13 avg-zval = 30.32 2024/11/23-17:49:59.158152 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273210.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:49:59.979775 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273210.sst): second pass time = 1.49's, 72.504'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.834'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3116.371'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 266.087'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38511.625'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.504'MB/sec zip pipeline throughput = 72.504'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 30.32 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 30.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0282 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.5135 value = 3.8295 dict = 1.00 all = 3.8937 } Zip/UnZip{ index = 0.0951 value = 0.2611 dict = 1.00 all = 0.2568 } ---------------------------- total value len = 10.336284 GB avg = 0.160 KB (by entry num) total key len = 1.779139 GB avg = 0.028 KB total ukey len = 1.239534 GB avg = 0.016 KB total ukey num = 0.077633221 Billion total entry num = 0.064440950 Billion write speed all = 0.377571428 MB/sec (with seq num) write speed all = 0.361505254 MB/sec (without seq num) 2024/11/23-17:50:00.022948 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273210: 931242 keys, 34470892 bytes, temperature: kWarm 2024/11/23-17:50:00.023032 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355400022996, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273210, "file_size": 34470892, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 28239200, "index_size": 1771520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355396, "oldest_key_time": 0, "file_creation_time": 1732355396, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273210, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:50:00.060480 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273211.sst) /dev/shm/ToplingTemp/Topling-138887-kp0tX9 2024/11/23-17:50:00.966444 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273211.sst): first pass time = 0.91's, 148.155'MB/sec 2024/11/23-17:50:02.334442 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273211.sst): old prealloc_size = 147639500, real_size = 35224480 2024/11/23-17:50:02.335048 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273211.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.46 avg-zval = 36.02 2024/11/23-17:50:02.336893 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273211.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:50:03.189243 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273211.sst): second pass time = 1.37's, 79.268'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.262'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3009.598'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 293.082'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34007.598'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.268'MB/sec zip pipeline throughput = 79.268'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 36.02 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 36.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5135 value = 3.2608 dict = 1.00 all = 3.3942 } Zip/UnZip{ index = 0.0951 value = 0.3067 dict = 1.00 all = 0.2946 } ---------------------------- total value len = 10.451125 GB avg = 0.160 KB (by entry num) total key len = 1.804975 GB avg = 0.028 KB total ukey len = 1.257988 GB avg = 0.016 KB total ukey num = 0.078555918 Billion total entry num = 0.065363647 Billion write speed all = 0.381917350 MB/sec (with seq num) write speed all = 0.365622763 MB/sec (without seq num) 2024/11/23-17:50:03.248850 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273211: 922697 keys, 39542870 bytes, temperature: kWarm 2024/11/23-17:50:03.248938 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355403248891, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273211, "file_size": 39542870, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33238352, "index_size": 1755264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355400, "oldest_key_time": 0, "file_creation_time": 1732355400, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273211, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:50:03.303443 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273212.sst) /dev/shm/ToplingTemp/Topling-138887-E61ltd 2024/11/23-17:50:04.393206 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273212.sst): first pass time = 1.09's, 123.169'MB/sec 2024/11/23-17:50:05.765705 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273212.sst): old prealloc_size = 147639500, real_size = 32232688 2024/11/23-17:50:05.766277 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273212.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0302 GB avg-val = 116.15 avg-zval = 32.45 2024/11/23-17:50:05.768084 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273212.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:50:06.761751 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273212.sst): second pass time = 1.37's, 78.848'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 36.723'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.166'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.25's, 148.534'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36795.469'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.848'MB/sec zip pipeline throughput = 78.848'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.15 avg-zval = 32.45 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.15 avg-zval = 32.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4434 value = 3.5791 dict = 1.00 all = 3.6695 } Zip/UnZip{ index = 0.0958 value = 0.2794 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 10.565790 GB avg = 0.159 KB (by entry num) total key len = 1.831045 GB avg = 0.028 KB total ukey len = 1.276609 GB avg = 0.016 KB total ukey num = 0.079487001 Billion total entry num = 0.066294730 Billion write speed all = 0.386259864 MB/sec (with seq num) write speed all = 0.369735006 MB/sec (without seq num) 2024/11/23-17:50:06.807702 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273212: 931083 keys, 36576128 bytes, temperature: kWarm 2024/11/23-17:50:06.807774 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355406807740, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273212, "file_size": 36576128, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30216608, "index_size": 1783104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355403, "oldest_key_time": 0, "file_creation_time": 1732355403, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273212, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:50:06.847234 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273213.sst) /dev/shm/ToplingTemp/Topling-138887-ARZnHd 2024/11/23-17:50:07.858739 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273213.sst): first pass time = 1.01's, 132.696'MB/sec 2024/11/23-17:50:09.197898 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273213.sst): old prealloc_size = 147639500, real_size = 32227408 2024/11/23-17:50:09.198438 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273213.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925987 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0302 GB avg-val = 116.95 avg-zval = 32.64 2024/11/23-17:50:09.199894 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273213.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:50:10.138005 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273213.sst): second pass time = 1.34's, 80.907'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 38.429'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3420.866'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.287'MB/sec write SST data time = 0.13's, 274.221'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35085.904'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.907'MB/sec zip pipeline throughput = 80.907'MB/sec entries = 925987 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.95 avg-zval = 32.64 usrkeys = 925987 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.98 avg-zval = 32.64 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 798378 bytes = 2894128 } seq expand size = 5588646 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4436 value = 3.5831 dict = 1.00 all = 3.7153 } Zip/UnZip{ index = 0.0958 value = 0.2791 dict = 1.00 all = 0.2692 } ---------------------------- total value len = 10.679669 GB avg = 0.159 KB (by entry num) total key len = 1.856973 GB avg = 0.028 KB total ukey len = 1.295129 GB avg = 0.016 KB total ukey num = 0.080412988 Billion total entry num = 0.067220717 Billion write speed all = 0.390574858 MB/sec (with seq num) write speed all = 0.373820948 MB/sec (without seq num) 2024/11/23-17:50:10.182110 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273213: 925987 keys, 36125980 bytes, temperature: kWarm 2024/11/23-17:50:10.182181 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355410182146, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273213, "file_size": 36125980, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 30222400, "index_size": 1773312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927636, "raw_average_key_size": 28, "raw_value_size": 108290194, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925987, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355406, "oldest_key_time": 0, "file_creation_time": 1732355406, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273213, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:50:10.224217 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273214.sst) /dev/shm/ToplingTemp/Topling-138887-oxOjZc 2024/11/23-17:50:11.229019 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273214.sst): first pass time = 1.00's, 133.582'MB/sec 2024/11/23-17:50:12.765732 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273214.sst): old prealloc_size = 147639500, real_size = 36917296 2024/11/23-17:50:12.766314 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273214.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926160 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.92 avg-zval = 37.70 2024/11/23-17:50:12.766546 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273214.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:50:13.476155 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273214.sst): second pass time = 1.54's, 70.502'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.937'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3168.375'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.396'MB/sec write SST data time = 0.13's, 306.026'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36243.467'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.502'MB/sec zip pipeline throughput = 70.502'MB/sec entries = 926160 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.92 avg-zval = 37.70 usrkeys = 926160 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.88 avg-zval = 37.70 TagRS{ kinds = RS_Key0_TagN bytes = 144800 } TagArray{ size = 126781 bytes = 396208 } seq expand size = 887467 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4444 value = 3.1017 dict = 1.00 all = 3.5028 } Zip/UnZip{ index = 0.0957 value = 0.3224 dict = 1.00 all = 0.2855 } ---------------------------- total value len = 10.788842 GB avg = 0.158 KB (by entry num) total key len = 1.882905 GB avg = 0.028 KB total ukey len = 1.313652 GB avg = 0.016 KB total ukey num = 0.081339148 Billion total entry num = 0.068146877 Billion write speed all = 0.394742965 MB/sec (with seq num) write speed all = 0.377759987 MB/sec (without seq num) 2024/11/23-17:50:13.532353 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273214: 926160 keys, 38316757 bytes, temperature: kWarm 2024/11/23-17:50:13.532430 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355413532394, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273214, "file_size": 38316757, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34912096, "index_size": 1773504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25932480, "raw_average_key_size": 28, "raw_value_size": 108285324, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926160, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355410, "oldest_key_time": 0, "file_creation_time": 1732355410, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273214, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:50:13.571821 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273215.sst) /dev/shm/ToplingTemp/Topling-138887-OS4OOa 2024/11/23-17:50:14.638654 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273215.sst): first pass time = 1.07's, 125.814'MB/sec 2024/11/23-17:50:16.028237 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273215.sst): old prealloc_size = 147639500, real_size = 36616224 2024/11/23-17:50:16.028782 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273215.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928863 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0346 GB avg-val = 116.50 avg-zval = 37.27 2024/11/23-17:50:16.030255 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273215.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:50:16.803237 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273215.sst): second pass time = 1.39's, 77.917'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 51.223'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3372.903'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.361'MB/sec write SST data time = 0.13's, 301.925'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34164.910'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.917'MB/sec zip pipeline throughput = 77.917'MB/sec entries = 928863 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.50 avg-zval = 37.27 usrkeys = 928863 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.67 avg-zval = 37.27 TagRS{ kinds = RS_Key0_TagN bytes = 145240 } TagArray{ size = 554258 bytes = 2078480 } seq expand size = 3879806 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.5147 value = 3.1259 dict = 1.00 all = 3.3811 } Zip/UnZip{ index = 0.0951 value = 0.3199 dict = 1.00 all = 0.2958 } ---------------------------- total value len = 10.900932 GB avg = 0.158 KB (by entry num) total key len = 1.908913 GB avg = 0.028 KB total ukey len = 1.332230 GB avg = 0.016 KB total ukey num = 0.082268011 Billion total entry num = 0.069075740 Billion write speed all = 0.399003550 MB/sec (with seq num) write speed all = 0.381790872 MB/sec (without seq num) 2024/11/23-17:50:16.856658 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273215: 928863 keys, 39696604 bytes, temperature: kWarm 2024/11/23-17:50:16.856725 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355416856690, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273215, "file_size": 39696604, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4160357968, "table_properties": {"data_size": 34617040, "index_size": 1766784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26008164, "raw_average_key_size": 28, "raw_value_size": 108209698, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928863, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355413, "oldest_key_time": 0, "file_creation_time": 1732355413, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273215, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:50:16.912600 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273216.sst) /dev/shm/ToplingTemp/Topling-138887-i8gPed 2024/11/23-17:50:18.072013 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273216.sst): first pass time = 1.16's, 115.768'MB/sec 2024/11/23-17:50:19.411743 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273216.sst): old prealloc_size = 147639500, real_size = 34550960 2024/11/23-17:50:19.412300 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273216.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925883 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 116.96 avg-zval = 35.16 2024/11/23-17:50:19.413064 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273216.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:50:20.257616 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273216.sst): second pass time = 1.34's, 80.879'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.552'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3305.528'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 299.656'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35924.650'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.879'MB/sec zip pipeline throughput = 80.879'MB/sec entries = 925883 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 35.16 usrkeys = 925883 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 35.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925883 bytes = 3124864 } seq expand size = 6481181 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5134 value = 3.3262 dict = 1.00 all = 3.4834 } Zip/UnZip{ index = 0.0951 value = 0.3006 dict = 1.00 all = 0.2871 } ---------------------------- total value len = 11.015706 GB avg = 0.157 KB (by entry num) total key len = 1.934838 GB avg = 0.028 KB total ukey len = 1.350747 GB avg = 0.016 KB total ukey num = 0.083193894 Billion total entry num = 0.070001623 Billion write speed all = 0.403342671 MB/sec (with seq num) write speed all = 0.385901154 MB/sec (without seq num) 2024/11/23-17:50:20.305647 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273216: 925883 keys, 38530618 bytes, temperature: kWarm 2024/11/23-17:50:20.305718 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355420305683, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273216, "file_size": 38530618, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4072856654, "largest_seqno": 4160357549, "table_properties": {"data_size": 32557984, "index_size": 1761344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924724, "raw_average_key_size": 28, "raw_value_size": 108293098, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925883, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355416, "oldest_key_time": 0, "file_creation_time": 1732355416, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273216, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:50:20.358762 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273217.sst) /dev/shm/ToplingTemp/Topling-138887-GQJBWb 2024/11/23-17:50:20.628183 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273217.sst): first pass time = 0.27's, 246.656'MB/sec 2024/11/23-17:50:21.301578 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273217.sst): old prealloc_size = 147639500, real_size = 17857056 2024/11/23-17:50:21.301874 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273217.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 459617 prefix = 4 raw-key = 0.0074 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0536 GB zip-val = 0.0169 GB avg-val = 116.57 avg-zval = 36.70 2024/11/23-17:50:21.302284 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273217.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.066 GB 2024/11/23-17:50:21.714259 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00/273217.sst): second pass time = 0.67's, 79.652'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.41's, 47.934'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3200.702'MB/sec (index lex order gen) rebuild zvType time = 0.35's, 0.329'MB/sec write SST data time = 0.06's, 310.998'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.541'MB, throughput = 18443.186'MB/sec zip my value time = 0.67's, unzip length = 0.054'GB zip my value throughput = 79.652'MB/sec zip pipeline throughput = 79.652'MB/sec entries = 459617 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.57 avg-zval = 36.70 usrkeys = 459617 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.57 avg-zval = 36.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 459617 bytes = 1493776 } seq expand size = 3217319 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0092 GB value = 0.0536 GB dict = 0.54 MB all = 0.0664 GB } __ZipSize{ index = 0.0009 GB value = 0.0169 GB dict = 0.54 MB all = 0.0198 GB } UnZip/Zip{ index = 10.5054 value = 3.1764 dict = 1.00 all = 3.3592 } Zip/UnZip{ index = 0.0952 value = 0.3148 dict = 1.00 all = 0.2977 } ---------------------------- total value len = 11.072499 GB avg = 0.157 KB (by entry num) total key len = 1.947707 GB avg = 0.028 KB total ukey len = 1.359940 GB avg = 0.016 KB total ukey num = 0.083653511 Billion total entry num = 0.070461240 Billion write speed all = 0.405493913 MB/sec (with seq num) write speed all = 0.387938675 MB/sec (without seq num) 2024/11/23-17:50:21.746516 139957433374272 [compaction_job.cc:1948] [default] [JOB 32342] Generated table #273217: 459617 keys, 19780101 bytes, temperature: kWarm 2024/11/23-17:50:21.746585 139957433374272 EVENT_LOG_v1 {"time_micros": 1732355421746547, "cf_name": "default", "job": 32342, "event": "table_file_creation", "file_number": 273217, "file_size": 19780101, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4116739047, "largest_seqno": 4158845171, "table_properties": {"data_size": 16866960, "index_size": 875008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 12869276, "raw_average_key_size": 28, "raw_value_size": 53576250, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 459617, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355420, "oldest_key_time": 0, "file_creation_time": 1732355420, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273217, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:50:21.764403 139957433374272 [compaction_job.cc:703] job-32342: subcompact[0], size: 0.748825 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32342/att-00 [273198,273199,273200,273201,273202,273203,273204,273205,273206,273207,273208,273209,273210,273211,273212,273213,273214,273215,273216,273217] 2024/11/23-17:50:21.922058 139957433374272 (Original Log Time 2024/11/23-17:49:11.793626) EVENT_LOG_v1 {"time_micros": 1732355351793577, "job": 32342, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [273193], "files_L3": [273148, 273149, 273150, 273151, 273153, 273154, 273155, 273156, 273157, 273158, 273159, 273160, 273161, 273162, 273163, 273164, 273165, 273166, 273167, 273116], "score": 0.989019, "input_data_size": 770023412, "oldest_snapshot_seqno": 4160378422} 2024/11/23-17:50:21.922062 139957433374272 (Original Log Time 2024/11/23-17:50:21.920483) [compaction_job.cc:2022] [default] [JOB 32342] Compacted 1@2 + 20@3 files to L3 => 748825444 bytes 2024/11/23-17:50:21.922063 139957433374272 (Original Log Time 2024/11/23-17:50:21.921970) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.0 rd, 10.7 wr, level 3, files in(1, 20) out(20 +0 blob) MB in(27.2, 707.2 +0.0 blob) out(714.1 +0.0 blob), read-write-amplify(53.3) write-amplify(26.3) OK, records in: 18170483, records dropped: 67407 output_compression: Snappy 2024/11/23-17:50:21.922067 139957433374272 (Original Log Time 2024/11/23-17:50:21.922032) EVENT_LOG_v1 {"time_micros": 1732355421922012, "cf": "default", "job": 32342, "event": "compaction_finished", "compaction_time_micros": 69970784, "compaction_time_cpu_micros": 29260063, "output_level": 3, "num_output_files": 20, "total_output_size": 748825444, "num_input_records": 18170483, "num_output_records": 18103076, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 542653250, "file_range_sync_nanos": 0, "file_fsync_nanos": 14414, "file_prepare_write_nanos": 60188, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-17:50:22.054406 139957433374272 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732355351,"executesMs":70261,"compactionJobId":32342,"attempt":0,"compactionInputRawBytes":2626155714,"compactionInputZipBytes":770023412,"compactionOutputRawBytes":0,"compactionOutputZipBytes":748825444,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-087302ecf7514b28bc40edcd9a796c51-0f03fb4f77b1e600-00"}}, response = 2024/11/23-17:50:22.054479 139957433374272 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32342/att-00: olev 3, work 70.261 s, result 7.685 ms, install 1.598 ms, input{raw 2.446 GiB zip 734.352 MiB}