2024/11/22-20:10:23.597427 140024818968128 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00: fork to child time = 0.009156 sec 2024/11/22-20:10:23.598423 140024818968128 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00 2024/11/22-20:10:23.605312 140024818968128 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00 2024/11/22-20:10:23.606455 140024818968128 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-20:10:23.606720 140024818968128 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-20:10:23.654323 140024818968128 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09785/att-00/MANIFEST-206347 2024/11/22-20:10:23.694271 140024818968128 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09785/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 225257, last_sequence is 3529332088, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 225056 2024/11/22-20:10:23.694281 140024818968128 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-20:10:23.694411 140024818968128 [version_set.cc:5721] Creating manifest 225257 2024/11/22-20:10:23.731636 140024818968128 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-20:10:23.751391 140024818968128 [compaction_job.cc:2437] [default] [JOB 9785] Compacting 1@2 + 17@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-20:10:23.751410 140024818968128 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 7396 Base level 2, inputs: [225188(27MB)], [225254(36MB) 225255(5584KB) 225145(33MB) 225146(34MB) 225147(34MB) 225148(12MB) 225109(35MB) 225152(36MB) 225153(36MB) 225154(35MB) 225155(35MB) 225156(35MB) 225157(35MB) 225158(35MB) 225159(36MB) 225160(36MB) 225237(35MB)] 2024/11/22-20:10:23.758741 140024818968128 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-20:10:23.758788 140024818968128 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-20:10:23.758809 140024818968128 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-20:10:23.758815 140024818968128 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-20:10:23.758825 140024818968128 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-20:10:24.363639 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225258.sst) /dev/shm/ToplingTemp/Topling-272731-QnX0mS 2024/11/22-20:10:24.913283 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225258.sst): first pass time = 0.55's, 244.212'MB/sec 2024/11/22-20:10:26.275016 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225258.sst): old prealloc_size = 147639500, real_size = 34179168 2024/11/22-20:10:26.275574 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225258.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931976 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0322 GB avg-val = 116.01 avg-zval = 34.52 2024/11/22-20:10:26.276404 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225258.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:10:27.184228 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225258.sst): second pass time = 1.36's, 79.450'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.116'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3298.726'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.299'MB/sec write SST data time = 0.13's, 296.737'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30964.945'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.450'MB/sec zip pipeline throughput = 79.450'MB/sec entries = 931976 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.01 avg-zval = 34.52 usrkeys = 931976 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.01 avg-zval = 34.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931976 bytes = 3261936 } seq expand size = 6523832 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5119 value = 3.3607 dict = 1.00 all = 3.5052 } Zip/UnZip{ index = 0.0951 value = 0.2976 dict = 1.00 all = 0.2853 } ---------------------------- total value len = 0.114646 GB avg = 0.123 KB (by entry num) total key len = 0.026095 GB avg = 0.028 KB total ukey len = 0.018640 GB avg = 0.020 KB total ukey num = 0.000931976 Billion total entry num = 0.000931976 Billion write speed all = 0.003459377 MB/sec (with seq num) write speed all = 0.003276116 MB/sec (without seq num) 2024/11/22-20:10:27.218493 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225258: 931976 keys, 38291546 bytes, temperature: kWarm 2024/11/22-20:10:27.218585 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277427218526, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225258, "file_size": 38291546, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3528876517, "table_properties": {"data_size": 32172816, "index_size": 1773184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26095328, "raw_average_key_size": 28, "raw_value_size": 108122456, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931976, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277424, "oldest_key_time": 0, "file_creation_time": 1732277424, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225258, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:10:27.255613 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225259.sst) /dev/shm/ToplingTemp/Topling-272731-QDmwSN 2024/11/22-20:10:28.212273 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225259.sst): first pass time = 0.96's, 140.304'MB/sec 2024/11/22-20:10:29.509186 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225259.sst): old prealloc_size = 147639500, real_size = 31230576 2024/11/22-20:10:29.509721 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225259.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920913 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.74 avg-zval = 31.73 2024/11/22-20:10:29.510526 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225259.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:10:30.370460 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225259.sst): second pass time = 1.30's, 83.651'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.000'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3470.082'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.314'MB/sec write SST data time = 0.13's, 275.197'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 17057.339'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.651'MB/sec zip pipeline throughput = 83.651'MB/sec entries = 920913 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.74 avg-zval = 31.73 usrkeys = 920913 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.74 avg-zval = 31.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920913 bytes = 3223216 } seq expand size = 6446391 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.3744 value = 3.7103 dict = 1.00 all = 3.8010 } Zip/UnZip{ index = 0.0964 value = 0.2695 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 0.229525 GB avg = 0.124 KB (by entry num) total key len = 0.051881 GB avg = 0.028 KB total ukey len = 0.037058 GB avg = 0.020 KB total ukey num = 0.001852889 Billion total entry num = 0.001852889 Billion write speed all = 0.006916308 MB/sec (with seq num) write speed all = 0.006551990 MB/sec (without seq num) 2024/11/22-20:10:30.402397 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225259: 920913 keys, 35311500 bytes, temperature: kWarm 2024/11/22-20:10:30.402460 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277430402429, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225259, "file_size": 35311500, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3278204396, "largest_seqno": 3528876297, "table_properties": {"data_size": 29224800, "index_size": 1775360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25785564, "raw_average_key_size": 28, "raw_value_size": 108432166, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920913, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277427, "oldest_key_time": 0, "file_creation_time": 1732277427, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225259, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:10:30.442375 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225260.sst) /dev/shm/ToplingTemp/Topling-272731-iRMROP 2024/11/22-20:10:31.265658 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225260.sst): first pass time = 0.82's, 163.036'MB/sec 2024/11/22-20:10:32.581541 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225260.sst): old prealloc_size = 147639500, real_size = 32231600 2024/11/22-20:10:32.582091 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225260.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924392 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0302 GB avg-val = 117.20 avg-zval = 32.70 2024/11/22-20:10:32.582882 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225260.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:10:33.474586 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225260.sst): second pass time = 1.32's, 82.369'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.674'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3344.411'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.302'MB/sec write SST data time = 0.13's, 284.369'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31767.746'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.369'MB/sec zip pipeline throughput = 82.369'MB/sec entries = 924392 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.20 avg-zval = 32.70 usrkeys = 924392 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.20 avg-zval = 32.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924392 bytes = 3235392 } seq expand size = 6470744 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4433 value = 3.5837 dict = 1.00 all = 3.6951 } Zip/UnZip{ index = 0.0958 value = 0.2790 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 0.344330 GB avg = 0.124 KB (by entry num) total key len = 0.077764 GB avg = 0.028 KB total ukey len = 0.055546 GB avg = 0.020 KB total ukey num = 0.002777281 Billion total entry num = 0.002777281 Billion write speed all = 0.010373320 MB/sec (with seq num) write speed all = 0.009827288 MB/sec (without seq num) 2024/11/22-20:10:33.507820 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225260: 924392 keys, 36323404 bytes, temperature: kWarm 2024/11/22-20:10:33.507890 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277433507854, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225260, "file_size": 36323404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3378289821, "largest_seqno": 3528876308, "table_properties": {"data_size": 30229984, "index_size": 1770304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25882976, "raw_average_key_size": 28, "raw_value_size": 108334876, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924392, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277430, "oldest_key_time": 0, "file_creation_time": 1732277430, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225260, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:10:33.541980 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225261.sst) /dev/shm/ToplingTemp/Topling-272731-5euD4R 2024/11/22-20:10:34.338704 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225261.sst): first pass time = 0.80's, 168.472'MB/sec 2024/11/22-20:10:35.662338 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225261.sst): old prealloc_size = 147639500, real_size = 31818848 2024/11/22-20:10:35.662902 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225261.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924117 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0298 GB avg-val = 117.24 avg-zval = 32.27 2024/11/22-20:10:35.663657 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225261.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:10:36.527728 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225261.sst): second pass time = 1.32's, 81.924'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.497'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3280.700'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.13's, 283.851'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32784.182'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.924'MB/sec zip pipeline throughput = 81.924'MB/sec entries = 924117 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.24 avg-zval = 32.27 usrkeys = 924117 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.24 avg-zval = 32.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924117 bytes = 3234432 } seq expand size = 6468819 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.4466 value = 3.6334 dict = 1.00 all = 3.7376 } Zip/UnZip{ index = 0.0957 value = 0.2752 dict = 1.00 all = 0.2675 } ---------------------------- total value len = 0.459142 GB avg = 0.124 KB (by entry num) total key len = 0.103639 GB avg = 0.028 KB total ukey len = 0.074028 GB avg = 0.020 KB total ukey num = 0.003701398 Billion total entry num = 0.003701398 Billion write speed all = 0.013829775 MB/sec (with seq num) write speed all = 0.013102110 MB/sec (without seq num) 2024/11/22-20:10:36.561131 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225261: 924117 keys, 35909926 bytes, temperature: kWarm 2024/11/22-20:10:36.561201 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277436561163, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225261, "file_size": 35909926, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774735, "largest_seqno": 3528876327, "table_properties": {"data_size": 29818448, "index_size": 1769216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25875276, "raw_average_key_size": 28, "raw_value_size": 108342566, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924117, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277433, "oldest_key_time": 0, "file_creation_time": 1732277433, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225261, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:10:36.597117 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225262.sst) /dev/shm/ToplingTemp/Topling-272731-KsqHvR 2024/11/22-20:10:37.548017 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225262.sst): first pass time = 0.95's, 141.155'MB/sec 2024/11/22-20:10:38.850511 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225262.sst): old prealloc_size = 147639500, real_size = 31701056 2024/11/22-20:10:38.851067 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225262.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924145 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 117.23 avg-zval = 32.15 2024/11/22-20:10:38.851844 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225262.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:10:39.723813 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225262.sst): second pass time = 1.30's, 83.240'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 40.986'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3269.061'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.310'MB/sec write SST data time = 0.13's, 280.175'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33479.655'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.240'MB/sec zip pipeline throughput = 83.240'MB/sec entries = 924145 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.23 avg-zval = 32.15 usrkeys = 924145 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.23 avg-zval = 32.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924145 bytes = 3234528 } seq expand size = 6469015 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5135 value = 3.6464 dict = 1.00 all = 3.7499 } Zip/UnZip{ index = 0.0951 value = 0.2742 dict = 1.00 all = 0.2667 } ---------------------------- total value len = 0.573953 GB avg = 0.124 KB (by entry num) total key len = 0.129515 GB avg = 0.028 KB total ukey len = 0.092511 GB avg = 0.020 KB total ukey num = 0.004625543 Billion total entry num = 0.004625543 Billion write speed all = 0.017285654 MB/sec (with seq num) write speed all = 0.016376381 MB/sec (without seq num) 2024/11/22-20:10:39.757135 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225262: 924145 keys, 35792314 bytes, temperature: kWarm 2024/11/22-20:10:39.757199 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277439757166, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225262, "file_size": 35792314, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3528876339, "table_properties": {"data_size": 29711792, "index_size": 1758016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876060, "raw_average_key_size": 28, "raw_value_size": 108341750, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924145, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277436, "oldest_key_time": 0, "file_creation_time": 1732277436, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225262, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:10:39.793952 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225263.sst) /dev/shm/ToplingTemp/Topling-272731-fQU1eS 2024/11/22-20:10:40.730440 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225263.sst): first pass time = 0.94's, 143.327'MB/sec 2024/11/22-20:10:41.998240 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225263.sst): old prealloc_size = 147639500, real_size = 31486992 2024/11/22-20:10:41.998830 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225263.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924141 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0295 GB avg-val = 117.24 avg-zval = 31.91 2024/11/22-20:10:41.999576 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225263.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:10:42.861075 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225263.sst): second pass time = 1.27's, 85.507'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.235'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3142.732'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.315'MB/sec write SST data time = 0.13's, 278.665'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 30034.808'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.507'MB/sec zip pipeline throughput = 85.507'MB/sec entries = 924141 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.24 avg-zval = 31.91 usrkeys = 924141 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.24 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924141 bytes = 3234512 } seq expand size = 6468987 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4446 value = 3.6743 dict = 1.00 all = 3.7725 } Zip/UnZip{ index = 0.0957 value = 0.2722 dict = 1.00 all = 0.2651 } ---------------------------- total value len = 0.688764 GB avg = 0.124 KB (by entry num) total key len = 0.155391 GB avg = 0.028 KB total ukey len = 0.110994 GB avg = 0.020 KB total ukey num = 0.005549684 Billion total entry num = 0.005549684 Billion write speed all = 0.020741021 MB/sec (with seq num) write speed all = 0.019650167 MB/sec (without seq num) 2024/11/22-20:10:42.892443 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225263: 924141 keys, 35578306 bytes, temperature: kWarm 2024/11/22-20:10:42.892506 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277442892474, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225263, "file_size": 35578306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774766, "largest_seqno": 3528876482, "table_properties": {"data_size": 29486144, "index_size": 1769600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25875948, "raw_average_key_size": 28, "raw_value_size": 108341910, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924141, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277439, "oldest_key_time": 0, "file_creation_time": 1732277439, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225263, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:10:42.929206 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225264.sst) /dev/shm/ToplingTemp/Topling-272731-L6pCjQ 2024/11/22-20:10:43.804062 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225264.sst): first pass time = 0.87's, 153.424'MB/sec 2024/11/22-20:10:45.110770 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225264.sst): old prealloc_size = 147639500, real_size = 31368336 2024/11/22-20:10:45.111305 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225264.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924118 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0294 GB avg-val = 117.24 avg-zval = 31.79 2024/11/22-20:10:45.112078 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225264.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:10:45.980552 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225264.sst): second pass time = 1.31's, 82.982'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 40.769'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3502.934'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.311'MB/sec write SST data time = 0.13's, 280.447'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31323.954'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.982'MB/sec zip pipeline throughput = 82.982'MB/sec entries = 924118 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.24 avg-zval = 31.79 usrkeys = 924118 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.24 avg-zval = 31.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924118 bytes = 3234432 } seq expand size = 6468826 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5155 value = 3.6877 dict = 1.00 all = 3.7851 } Zip/UnZip{ index = 0.0951 value = 0.2712 dict = 1.00 all = 0.2642 } ---------------------------- total value len = 0.803575 GB avg = 0.124 KB (by entry num) total key len = 0.181266 GB avg = 0.028 KB total ukey len = 0.129476 GB avg = 0.020 KB total ukey num = 0.006473802 Billion total entry num = 0.006473802 Billion write speed all = 0.024195861 MB/sec (with seq num) write speed all = 0.022923459 MB/sec (without seq num) 2024/11/22-20:10:46.012718 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225264: 924118 keys, 35459702 bytes, temperature: kWarm 2024/11/22-20:10:46.012790 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277446012749, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225264, "file_size": 35459702, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774786, "largest_seqno": 3528876368, "table_properties": {"data_size": 29379520, "index_size": 1757632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25875304, "raw_average_key_size": 28, "raw_value_size": 108342540, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924118, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277442, "oldest_key_time": 0, "file_creation_time": 1732277442, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225264, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:10:46.050122 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225265.sst) /dev/shm/ToplingTemp/Topling-272731-N7M9fQ 2024/11/22-20:10:46.981656 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225265.sst): first pass time = 0.93's, 144.089'MB/sec 2024/11/22-20:10:48.282556 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225265.sst): old prealloc_size = 147639500, real_size = 31126096 2024/11/22-20:10:48.283088 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225265.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923890 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0291 GB avg-val = 117.27 avg-zval = 31.54 2024/11/22-20:10:48.283886 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225265.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:10:49.133880 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225265.sst): second pass time = 1.30's, 83.328'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.502'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3443.282'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 268.951'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33170.748'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.328'MB/sec zip pipeline throughput = 83.328'MB/sec entries = 923890 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.27 avg-zval = 31.54 usrkeys = 923890 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.20 avg-zval = 31.54 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 914434 bytes = 3200528 } seq expand size = 6401038 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.5133 value = 3.7186 dict = 1.00 all = 3.7988 } Zip/UnZip{ index = 0.0951 value = 0.2689 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 0.918325 GB avg = 0.124 KB (by entry num) total key len = 0.207135 GB avg = 0.028 KB total ukey len = 0.147954 GB avg = 0.020 KB total ukey num = 0.007397692 Billion total entry num = 0.007397692 Billion write speed all = 0.027648484 MB/sec (with seq num) write speed all = 0.026194608 MB/sec (without seq num) 2024/11/22-20:10:49.165180 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225265: 923890 keys, 35331878 bytes, temperature: kWarm 2024/11/22-20:10:49.165243 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277449165210, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225265, "file_size": 35331878, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3528876621, "table_properties": {"data_size": 29137344, "index_size": 1757568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868920, "raw_average_key_size": 28, "raw_value_size": 108348944, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923890, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277446, "oldest_key_time": 0, "file_creation_time": 1732277446, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225265, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:10:49.203915 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225266.sst) /dev/shm/ToplingTemp/Topling-272731-7yvQsR 2024/11/22-20:10:50.005411 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225266.sst): first pass time = 0.80's, 167.469'MB/sec 2024/11/22-20:10:51.415311 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225266.sst): old prealloc_size = 147639500, real_size = 34969120 2024/11/22-20:10:51.415926 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225266.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081533 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.10 avg-zval = 30.17 2024/11/22-20:10:51.415972 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225266.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:10:52.041632 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225266.sst): second pass time = 1.41's, 73.781'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.063'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3485.581'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.556'MB/sec write SST data time = 0.14's, 256.174'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.029'MB, throughput = 28916.353'MB/sec zip my value time = 1.41's, unzip length = 0.104'GB zip my value throughput = 73.781'MB/sec zip pipeline throughput = 73.781'MB/sec entries = 1081533 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.10 avg-zval = 30.17 usrkeys = 1081533 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.10 avg-zval = 30.17 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.03 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4476 value = 3.1854 dict = 1.00 all = 3.7555 } Zip/UnZip{ index = 0.0957 value = 0.3139 dict = 1.00 all = 0.2663 } ---------------------------- total value len = 1.022260 GB avg = 0.121 KB (by entry num) total key len = 0.237418 GB avg = 0.028 KB total ukey len = 0.169584 GB avg = 0.020 KB total ukey num = 0.008479225 Billion total entry num = 0.008479225 Billion write speed all = 0.030943519 MB/sec (with seq num) write speed all = 0.029277207 MB/sec (without seq num) 2024/11/22-20:10:52.073172 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225266: 1081533 keys, 35738767 bytes, temperature: kWarm 2024/11/22-20:10:52.073236 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277452073204, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225266, "file_size": 35738767, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32628176, "index_size": 2070400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30282924, "raw_average_key_size": 28, "raw_value_size": 103934851, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081533, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277449, "oldest_key_time": 0, "file_creation_time": 1732277449, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225266, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:10:52.110175 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225267.sst) /dev/shm/ToplingTemp/Topling-272731-qMfmcR 2024/11/22-20:10:52.933168 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225267.sst): first pass time = 0.82's, 163.093'MB/sec 2024/11/22-20:10:54.292434 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225267.sst): old prealloc_size = 147639500, real_size = 34459920 2024/11/22-20:10:54.292987 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225267.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921869 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0325 GB avg-val = 117.59 avg-zval = 35.23 2024/11/22-20:10:54.293563 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225267.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:10:55.150591 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225267.sst): second pass time = 1.36's, 79.792'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.876'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3354.023'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 291.610'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33344.003'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.792'MB/sec zip pipeline throughput = 79.792'MB/sec entries = 921869 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.59 avg-zval = 35.23 usrkeys = 921869 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.33 avg-zval = 35.23 TagRS{ kinds = RS_Key0_TagN bytes = 144160 } TagArray{ size = 624455 bytes = 2185600 } seq expand size = 4371185 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5132 value = 3.3381 dict = 1.00 all = 3.5647 } Zip/UnZip{ index = 0.0951 value = 0.2996 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 1.135036 GB avg = 0.121 KB (by entry num) total key len = 0.263231 GB avg = 0.028 KB total ukey len = 0.188022 GB avg = 0.020 KB total ukey num = 0.009401094 Billion total entry num = 0.009401094 Billion write speed all = 0.034345280 MB/sec (with seq num) write speed all = 0.032497947 MB/sec (without seq num) 2024/11/22-20:10:55.185521 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225267: 921869 keys, 37651972 bytes, temperature: kWarm 2024/11/22-20:10:55.185584 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277455185552, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225267, "file_size": 37651972, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3501383006, "table_properties": {"data_size": 32475520, "index_size": 1753728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25812332, "raw_average_key_size": 28, "raw_value_size": 108405398, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921869, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277452, "oldest_key_time": 0, "file_creation_time": 1732277452, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225267, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:10:55.227671 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225268.sst) /dev/shm/ToplingTemp/Topling-272731-FY5XUQ 2024/11/22-20:10:56.170879 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225268.sst): first pass time = 0.94's, 142.306'MB/sec 2024/11/22-20:10:57.485041 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225268.sst): old prealloc_size = 147639500, real_size = 33432752 2024/11/22-20:10:57.485595 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225268.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923562 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0314 GB avg-val = 117.33 avg-zval = 34.03 2024/11/22-20:10:57.486341 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225268.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:10:58.423683 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225268.sst): second pass time = 1.31's, 82.522'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.853'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3323.460'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.285'MB/sec write SST data time = 0.13's, 292.262'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31017.532'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.522'MB/sec zip pipeline throughput = 82.522'MB/sec entries = 923562 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.33 avg-zval = 34.03 usrkeys = 923562 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.33 avg-zval = 34.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923562 bytes = 3117040 } seq expand size = 6464934 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4426 value = 3.4473 dict = 1.00 all = 3.5880 } Zip/UnZip{ index = 0.0958 value = 0.2901 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 1.249859 GB avg = 0.121 KB (by entry num) total key len = 0.289090 GB avg = 0.028 KB total ukey len = 0.206493 GB avg = 0.020 KB total ukey num = 0.010324656 Billion total entry num = 0.010324656 Billion write speed all = 0.037797798 MB/sec (with seq num) write speed all = 0.035769145 MB/sec (without seq num) 2024/11/22-20:10:58.457184 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225268: 923562 keys, 37407448 bytes, temperature: kWarm 2024/11/22-20:10:58.457251 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277458457218, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225268, "file_size": 37407448, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3394785136, "largest_seqno": 3528876553, "table_properties": {"data_size": 31432864, "index_size": 1768832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25859736, "raw_average_key_size": 28, "raw_value_size": 108358124, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923562, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277455, "oldest_key_time": 0, "file_creation_time": 1732277455, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225268, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:10:58.495216 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225269.sst) /dev/shm/ToplingTemp/Topling-272731-AgqYoP 2024/11/22-20:10:59.437824 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225269.sst): first pass time = 0.94's, 142.397'MB/sec 2024/11/22-20:11:00.779517 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225269.sst): old prealloc_size = 147639500, real_size = 33203552 2024/11/22-20:11:00.780077 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225269.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923670 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0312 GB avg-val = 117.31 avg-zval = 33.79 2024/11/22-20:11:00.781958 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225269.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:11:01.703881 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225269.sst): second pass time = 1.34's, 80.802'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.346'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3282.140'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 290.246'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 29655.962'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.802'MB/sec zip pipeline throughput = 80.802'MB/sec entries = 923670 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.31 avg-zval = 33.79 usrkeys = 923670 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.31 avg-zval = 33.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923670 bytes = 3232864 } seq expand size = 6465690 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5135 value = 3.4712 dict = 1.00 all = 3.5990 } Zip/UnZip{ index = 0.0951 value = 0.2881 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 1.364680 GB avg = 0.121 KB (by entry num) total key len = 0.314953 GB avg = 0.028 KB total ukey len = 0.224967 GB avg = 0.020 KB total ukey num = 0.011248326 Billion total entry num = 0.011248326 Billion write speed all = 0.041249771 MB/sec (with seq num) write speed all = 0.039039808 MB/sec (without seq num) 2024/11/22-20:11:01.737924 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225269: 923670 keys, 37293604 bytes, temperature: kWarm 2024/11/22-20:11:01.737989 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277461737957, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225269, "file_size": 37293604, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3365258144, "largest_seqno": 3528876787, "table_properties": {"data_size": 31215312, "index_size": 1757120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25862760, "raw_average_key_size": 28, "raw_value_size": 108355064, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923670, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277458, "oldest_key_time": 0, "file_creation_time": 1732277458, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225269, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:11:01.775099 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225270.sst) /dev/shm/ToplingTemp/Topling-272731-QtbzPO 2024/11/22-20:11:02.594045 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225270.sst): first pass time = 0.82's, 163.900'MB/sec 2024/11/22-20:11:03.968198 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225270.sst): old prealloc_size = 147639500, real_size = 34336176 2024/11/22-20:11:03.968732 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225270.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927483 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.71 avg-zval = 34.86 2024/11/22-20:11:03.970634 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225270.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:11:04.886948 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225270.sst): second pass time = 1.37's, 78.811'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.837'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3507.843'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 296.554'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31017.223'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.811'MB/sec zip pipeline throughput = 78.811'MB/sec entries = 927483 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.71 avg-zval = 34.86 usrkeys = 927483 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.71 avg-zval = 34.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927483 bytes = 3246208 } seq expand size = 6492381 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4446 value = 3.3484 dict = 1.00 all = 3.4919 } Zip/UnZip{ index = 0.0957 value = 0.2986 dict = 1.00 all = 0.2864 } ---------------------------- total value len = 1.479421 GB avg = 0.122 KB (by entry num) total key len = 0.340923 GB avg = 0.028 KB total ukey len = 0.243516 GB avg = 0.020 KB total ukey num = 0.012175809 Billion total entry num = 0.012175809 Billion write speed all = 0.044701950 MB/sec (with seq num) write speed all = 0.042309951 MB/sec (without seq num) 2024/11/22-20:11:04.922832 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225270: 927483 keys, 38437010 bytes, temperature: kWarm 2024/11/22-20:11:04.922900 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277464922864, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225270, "file_size": 38437010, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3528876963, "table_properties": {"data_size": 32328096, "index_size": 1776000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25969524, "raw_average_key_size": 28, "raw_value_size": 108248318, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927483, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277461, "oldest_key_time": 0, "file_creation_time": 1732277461, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225270, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:11:04.960019 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225271.sst) /dev/shm/ToplingTemp/Topling-272731-9OaDoP 2024/11/22-20:11:05.861888 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225271.sst): first pass time = 0.90's, 148.829'MB/sec 2024/11/22-20:11:07.210944 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225271.sst): old prealloc_size = 147639500, real_size = 33682640 2024/11/22-20:11:07.211493 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225271.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927923 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0317 GB avg-val = 116.64 avg-zval = 34.13 2024/11/22-20:11:07.213338 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225271.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:11:08.140523 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225271.sst): second pass time = 1.35's, 80.269'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.647'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3388.397'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 292.738'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31567.182'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.269'MB/sec zip pipeline throughput = 80.269'MB/sec entries = 927923 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.64 avg-zval = 34.13 usrkeys = 927923 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.64 avg-zval = 34.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927923 bytes = 3247744 } seq expand size = 6495461 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4439 value = 3.4172 dict = 1.00 all = 3.5522 } Zip/UnZip{ index = 0.0957 value = 0.2926 dict = 1.00 all = 0.2815 } ---------------------------- total value len = 1.594152 GB avg = 0.122 KB (by entry num) total key len = 0.366904 GB avg = 0.028 KB total ukey len = 0.262075 GB avg = 0.020 KB total ukey num = 0.013103732 Billion total entry num = 0.013103732 Billion write speed all = 0.048153581 MB/sec (with seq num) write speed all = 0.045579493 MB/sec (without seq num) 2024/11/22-20:11:08.172632 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225271: 927923 keys, 37784280 bytes, temperature: kWarm 2024/11/22-20:11:08.172697 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277468172666, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225271, "file_size": 37784280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3528876448, "table_properties": {"data_size": 31673536, "index_size": 1776960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25981844, "raw_average_key_size": 28, "raw_value_size": 108235954, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927923, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277464, "oldest_key_time": 0, "file_creation_time": 1732277464, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225271, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:11:08.209260 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225272.sst) /dev/shm/ToplingTemp/Topling-272731-w7ZjqS 2024/11/22-20:11:09.100106 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225272.sst): first pass time = 0.89's, 150.671'MB/sec 2024/11/22-20:11:10.431757 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225272.sst): old prealloc_size = 147639500, real_size = 31885312 2024/11/22-20:11:10.432304 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225272.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924012 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0299 GB avg-val = 117.26 avg-zval = 32.36 2024/11/22-20:11:10.434169 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225272.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-20:11:11.314050 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225272.sst): second pass time = 1.33's, 81.403'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.777'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3386.946'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.307'MB/sec write SST data time = 0.13's, 279.001'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 31104.564'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.403'MB/sec zip pipeline throughput = 81.403'MB/sec entries = 924012 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.26 avg-zval = 32.36 usrkeys = 924012 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.26 avg-zval = 32.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924012 bytes = 3234064 } seq expand size = 6468084 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5143 value = 3.6240 dict = 1.00 all = 3.7307 } Zip/UnZip{ index = 0.0951 value = 0.2759 dict = 1.00 all = 0.2680 } ---------------------------- total value len = 1.708966 GB avg = 0.122 KB (by entry num) total key len = 0.392777 GB avg = 0.028 KB total ukey len = 0.280555 GB avg = 0.020 KB total ukey num = 0.014027744 Billion total entry num = 0.014027744 Billion write speed all = 0.051604089 MB/sec (with seq num) write speed all = 0.048848703 MB/sec (without seq num) 2024/11/22-20:11:11.346348 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225272: 924012 keys, 35976414 bytes, temperature: kWarm 2024/11/22-20:11:11.346413 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277471346380, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225272, "file_size": 35976414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701165, "largest_seqno": 3528876655, "table_properties": {"data_size": 29896496, "index_size": 1757632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25872336, "raw_average_key_size": 28, "raw_value_size": 108345460, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924012, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277468, "oldest_key_time": 0, "file_creation_time": 1732277468, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225272, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:11:11.384937 140024818968128 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225273.sst) /dev/shm/ToplingTemp/Topling-272731-71EhZO 2024/11/22-20:11:11.708314 140024818968128 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225273.sst): first pass time = 0.32's, 247.753'MB/sec 2024/11/22-20:11:12.501291 140024818968128 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225273.sst): old prealloc_size = 147639500, real_size = 20606512 2024/11/22-20:11:12.501652 140024818968128 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225273.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 549913 prefix = 4 raw-key = 0.0088 GB zip-key = 0.0010 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0647 GB zip-val = 0.0194 GB avg-val = 117.67 avg-zval = 35.32 2024/11/22-20:11:12.502132 140024818968128 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225273.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0014 GB, waited 0.000 sec, Key+Value = 0.080 GB 2024/11/22-20:11:13.074037 140024818968128 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00/225273.sst): second pass time = 0.79's, 81.673'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.57's, 40.242'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3087.004'MB/sec (index lex order gen) rebuild zvType time = 0.50's, 0.277'MB/sec write SST data time = 0.08's, 301.032'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.650'MB, throughput = 21342.591'MB/sec zip my value time = 0.79's, unzip length = 0.065'GB zip my value throughput = 81.673'MB/sec zip pipeline throughput = 81.673'MB/sec entries = 549913 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.67 avg-zval = 35.32 usrkeys = 549913 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.67 avg-zval = 35.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 549913 bytes = 1924704 } seq expand size = 3849391 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0110 GB value = 0.0647 GB dict = 0.65 MB all = 0.0801 GB } __ZipSize{ index = 0.0010 GB value = 0.0194 GB dict = 0.65 MB all = 0.0230 GB } UnZip/Zip{ index = 10.5048 value = 3.3318 dict = 1.00 all = 3.4757 } Zip/UnZip{ index = 0.0952 value = 0.3001 dict = 1.00 all = 0.2877 } ---------------------------- total value len = 1.777524 GB avg = 0.122 KB (by entry num) total key len = 0.408174 GB avg = 0.028 KB total ukey len = 0.291553 GB avg = 0.020 KB total ukey num = 0.014577657 Billion total entry num = 0.014577657 Billion write speed all = 0.053663149 MB/sec (with seq num) write speed all = 0.050799871 MB/sec (without seq num) 2024/11/22-20:11:13.097440 140024818968128 [compaction_job.cc:1948] [default] [JOB 9785] Generated table #225273: 549913 keys, 23047849 bytes, temperature: kWarm 2024/11/22-20:11:13.097503 140024818968128 EVENT_LOG_v1 {"time_micros": 1732277473097473, "cf_name": "default", "job": 9785, "event": "table_file_creation", "file_number": 225273, "file_size": 23047849, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3388810268, "largest_seqno": 3528875920, "table_properties": {"data_size": 19421856, "index_size": 1046976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 15397564, "raw_average_key_size": 28, "raw_value_size": 64709338, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 549913, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732277471, "oldest_key_time": 0, "file_creation_time": 1732277471, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 225273, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-20:11:13.118320 140024818968128 [compaction_job.cc:703] job-09785: subcompact[0], size: 0.571336 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09785/att-00 [225258,225259,225260,225261,225262,225263,225264,225265,225266,225267,225268,225269,225270,225271,225272,225273] 2024/11/22-20:11:13.217154 140024818968128 (Original Log Time 2024/11/22-20:10:23.751468) EVENT_LOG_v1 {"time_micros": 1732277423751426, "job": 9785, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [225188], "files_L3": [225254, 225255, 225145, 225146, 225147, 225148, 225109, 225152, 225153, 225154, 225155, 225156, 225157, 225158, 225159, 225160, 225237], "score": 0.999088, "input_data_size": 606491825, "oldest_snapshot_seqno": 3529332088} 2024/11/22-20:11:13.217157 140024818968128 (Original Log Time 2024/11/22-20:11:13.216324) [compaction_job.cc:2022] [default] [JOB 9785] Compacted 1@2 + 17@3 files to L3 => 571335920 bytes 2024/11/22-20:11:13.217159 140024818968128 (Original Log Time 2024/11/22-20:11:13.217084) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 12.3 rd, 11.6 wr, level 3, files in(1, 17) out(16 +0 blob) MB in(27.1, 551.3 +0.0 blob) out(544.9 +0.0 blob), read-write-amplify(41.5) write-amplify(20.1) OK, records in: 14668907, records dropped: 91250 output_compression: Snappy 2024/11/22-20:11:13.217165 140024818968128 (Original Log Time 2024/11/22-20:11:13.217132) EVENT_LOG_v1 {"time_micros": 1732277473217109, "cf": "default", "job": 9785, "event": "compaction_finished", "compaction_time_micros": 49366860, "compaction_time_cpu_micros": 22301617, "output_level": 3, "num_output_files": 16, "total_output_size": 571335920, "num_input_records": 14668907, "num_output_records": 14577657, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 366000755, "file_range_sync_nanos": 0, "file_fsync_nanos": 15434, "file_prepare_write_nanos": 38217, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/22-20:11:13.334817 140024818968128 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732277423,"executesMs":49613,"compactionJobId":9785,"attempt":0,"compactionInputRawBytes":2106331611,"compactionInputZipBytes":606491825,"compactionOutputRawBytes":0,"compactionOutputZipBytes":571335920,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-b1f6799774b599ed65fc5e2f371349ad-e1943cb3f58b7e66-00"}}, response = 2024/11/22-20:11:13.334894 140024818968128 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09785/att-00: olev 3, work 49.614 s, result 5.019 ms, install 0.875 ms, input{raw 1.962 GiB zip 578.396 MiB}