2024/11/23-05:45:37.676232 140024793790016 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00: fork to child time = 0.008655 sec 2024/11/23-05:45:37.677123 140024793790016 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00 2024/11/23-05:45:37.683803 140024793790016 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00 2024/11/23-05:45:37.684923 140024793790016 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-05:45:37.685187 140024793790016 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-05:45:37.732596 140024793790016 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20439/att-00/MANIFEST-206347 2024/11/23-05:45:37.750496 140024793790016 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20439/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 247202, last_sequence is 3818200915, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 247001 2024/11/23-05:45:37.750501 140024793790016 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-05:45:37.750797 140024793790016 [version_set.cc:5721] Creating manifest 247202 2024/11/23-05:45:37.780250 140024793790016 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-05:45:37.799887 140024793790016 [compaction_job.cc:2437] [default] [JOB 20439] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-05:45:37.799903 140024793790016 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 15280 Base level 2, inputs: [247164(27MB)], [247199(35MB) 247200(7915KB) 247104(35MB) 247105(34MB) 247106(36MB) 247107(37MB) 247108(33MB) 247109(35MB) 247110(35MB) 247111(34MB) 247112(35MB) 247113(35MB) 247114(33MB) 247115(34MB) 247178(36MB)] 2024/11/23-05:45:37.805866 140024793790016 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-05:45:37.805897 140024793790016 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-05:45:37.805917 140024793790016 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-05:45:37.805922 140024793790016 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-05:45:37.805932 140024793790016 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-05:45:38.280510 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247203.sst) /dev/shm/ToplingTemp/Topling-425412-YVv8eT 2024/11/23-05:45:38.845308 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247203.sst): first pass time = 0.56's, 237.659'MB/sec 2024/11/23-05:45:40.374676 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247203.sst): old prealloc_size = 147639500, real_size = 32670448 2024/11/23-05:45:40.375267 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247203.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928483 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.56 avg-zval = 33.03 2024/11/23-05:45:40.376171 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247203.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:45:41.480141 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247203.sst): second pass time = 1.53's, 70.804'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 33.370'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3117.826'MB/sec (index lex order gen) rebuild zvType time = 0.96's, 0.243'MB/sec write SST data time = 0.15's, 248.692'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33903.338'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.804'MB/sec zip pipeline throughput = 70.804'MB/sec entries = 928483 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.56 avg-zval = 33.03 usrkeys = 928483 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.56 avg-zval = 33.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928483 bytes = 3365760 } seq expand size = 6499381 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5154 value = 3.5283 dict = 1.00 all = 3.6384 } Zip/UnZip{ index = 0.0951 value = 0.2834 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 0.114720 GB avg = 0.124 KB (by entry num) total key len = 0.025998 GB avg = 0.028 KB total ukey len = 0.018570 GB avg = 0.020 KB total ukey num = 0.000928483 Billion total entry num = 0.000928483 Billion write speed all = 0.001871280 MB/sec (with seq num) write speed all = 0.001772503 MB/sec (without seq num) 2024/11/23-05:45:41.516613 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247203: 928483 keys, 36889520 bytes, temperature: kWarm 2024/11/23-05:45:41.516712 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311941516657, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247203, "file_size": 36889520, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3817785814, "table_properties": {"data_size": 30672160, "index_size": 1765952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997524, "raw_average_key_size": 28, "raw_value_size": 108220338, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928483, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311938, "oldest_key_time": 0, "file_creation_time": 1732311938, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247203, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:45:41.551161 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247204.sst) /dev/shm/ToplingTemp/Topling-425412-3BO3EQ 2024/11/23-05:45:42.624124 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247204.sst): first pass time = 1.07's, 125.095'MB/sec 2024/11/23-05:45:44.115272 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247204.sst): old prealloc_size = 147639500, real_size = 33079552 2024/11/23-05:45:44.115908 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247204.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928204 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.60 avg-zval = 33.49 2024/11/23-05:45:44.116706 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247204.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:45:45.057450 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247204.sst): second pass time = 1.49's, 72.620'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 39.464'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2873.551'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.288'MB/sec write SST data time = 0.14's, 274.668'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31428.472'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.620'MB/sec zip pipeline throughput = 72.620'MB/sec entries = 928204 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.60 avg-zval = 33.49 usrkeys = 928204 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.60 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928204 bytes = 3248736 } seq expand size = 6497428 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5134 value = 3.4821 dict = 1.00 all = 3.6098 } Zip/UnZip{ index = 0.0951 value = 0.2872 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 0.229445 GB avg = 0.124 KB (by entry num) total key len = 0.051987 GB avg = 0.028 KB total ukey len = 0.037134 GB avg = 0.020 KB total ukey num = 0.001856687 Billion total entry num = 0.001856687 Billion write speed all = 0.003742356 MB/sec (with seq num) write speed all = 0.003544841 MB/sec (without seq num) 2024/11/23-05:45:45.090879 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247204: 928204 keys, 37181884 bytes, temperature: kWarm 2024/11/23-05:45:45.090942 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311945090911, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247204, "file_size": 37181884, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3587898319, "largest_seqno": 3817786851, "table_properties": {"data_size": 31081584, "index_size": 1765760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989712, "raw_average_key_size": 28, "raw_value_size": 108228132, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928204, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311941, "oldest_key_time": 0, "file_creation_time": 1732311941, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247204, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:45:45.125171 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247205.sst) /dev/shm/ToplingTemp/Topling-425412-mSxCMQ 2024/11/23-05:45:46.123802 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247205.sst): first pass time = 1.00's, 134.407'MB/sec 2024/11/23-05:45:47.998350 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247205.sst): old prealloc_size = 147639500, real_size = 34150640 2024/11/23-05:45:47.998952 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247205.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931985 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0321 GB avg-val = 116.01 avg-zval = 34.48 2024/11/23-05:45:47.999458 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247205.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:45:49.016314 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247205.sst): second pass time = 1.87's, 57.720'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 36.191'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3062.011'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.288'MB/sec write SST data time = 0.21's, 176.869'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32601.693'MB/sec zip my value time = 1.87's, unzip length = 0.108'GB zip my value throughput = 57.720'MB/sec zip pipeline throughput = 127.988'MB/sec entries = 931985 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.01 avg-zval = 34.48 usrkeys = 931985 avg-key = 20.00 avg-zkey = 1.91 avg-val = 119.92 avg-zval = 34.48 TagRS{ kinds = RS_Key0_TagN bytes = 145720 } TagArray{ size = 519696 bytes = 1689024 } seq expand size = 3637872 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4456 value = 3.3648 dict = 1.00 all = 3.6432 } Zip/UnZip{ index = 0.0957 value = 0.2972 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 0.341205 GB avg = 0.122 KB (by entry num) total key len = 0.078083 GB avg = 0.028 KB total ukey len = 0.055773 GB avg = 0.020 KB total ukey num = 0.002788672 Billion total entry num = 0.002788672 Billion write speed all = 0.005575202 MB/sec (with seq num) write speed all = 0.005278558 MB/sec (without seq num) 2024/11/23-05:45:49.049638 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247205: 931985 keys, 36840329 bytes, temperature: kWarm 2024/11/23-05:45:49.049787 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311949049671, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247205, "file_size": 36840329, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3817787155, "table_properties": {"data_size": 32133024, "index_size": 1784448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26095580, "raw_average_key_size": 28, "raw_value_size": 108122230, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931985, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311945, "oldest_key_time": 0, "file_creation_time": 1732311945, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247205, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:45:49.085405 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247206.sst) /dev/shm/ToplingTemp/Topling-425412-0nbS9O 2024/11/23-05:45:49.912183 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247206.sst): first pass time = 0.83's, 162.345'MB/sec 2024/11/23-05:45:51.455692 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247206.sst): old prealloc_size = 147639500, real_size = 35441504 2024/11/23-05:45:51.456365 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247206.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1080669 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1040 GB zip-val = 0.0331 GB avg-val = 96.20 avg-zval = 30.63 2024/11/23-05:45:51.456596 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247206.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:45:52.289997 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247206.sst): second pass time = 1.54's, 67.385'MB/sec, value only(77.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.027'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3184.896'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.391'MB/sec write SST data time = 0.14's, 257.415'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.027'MB, throughput = 30563.958'MB/sec zip my value time = 1.54's, unzip length = 0.104'GB zip my value throughput = 67.385'MB/sec zip pipeline throughput = 67.385'MB/sec entries = 1080669 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.20 avg-zval = 30.63 usrkeys = 1080669 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.88 avg-zval = 30.63 TagRS{ kinds = RS_Key0_TagN bytes = 168960 } TagArray{ size = 104555 bytes = 352896 } seq expand size = 731885 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0216 GB value = 0.1040 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0331 GB dict = 1.03 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4473 value = 3.1405 dict = 1.00 all = 3.6541 } Zip/UnZip{ index = 0.0957 value = 0.3184 dict = 1.00 all = 0.2737 } ---------------------------- total value len = 0.445896 GB avg = 0.115 KB (by entry num) total key len = 0.108342 GB avg = 0.028 KB total ukey len = 0.077387 GB avg = 0.020 KB total ukey num = 0.003869341 Billion total entry num = 0.003869341 Billion write speed all = 0.007369284 MB/sec (with seq num) write speed all = 0.006957702 MB/sec (without seq num) 2024/11/23-05:45:52.347264 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247206: 1080669 keys, 36731028 bytes, temperature: kWarm 2024/11/23-05:45:52.347314 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311952347287, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247206, "file_size": 36731028, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3401489948, "table_properties": {"data_size": 33102352, "index_size": 2068800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30258732, "raw_average_key_size": 28, "raw_value_size": 103959071, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1080669, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311949, "oldest_key_time": 0, "file_creation_time": 1732311949, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247206, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:45:52.390154 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247207.sst) /dev/shm/ToplingTemp/Topling-425412-aZUu0R 2024/11/23-05:45:53.289593 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247207.sst): first pass time = 0.90's, 149.228'MB/sec 2024/11/23-05:45:54.705738 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247207.sst): old prealloc_size = 147639500, real_size = 34090224 2024/11/23-05:45:54.706456 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247207.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930824 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0321 GB avg-val = 116.19 avg-zval = 34.47 2024/11/23-05:45:54.707327 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247207.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:45:55.744100 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247207.sst): second pass time = 1.42's, 76.413'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 36.901'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2529.681'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.265'MB/sec write SST data time = 0.16's, 240.095'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31296.756'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.413'MB/sec zip pipeline throughput = 76.413'MB/sec entries = 930824 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.19 avg-zval = 34.47 usrkeys = 930824 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.19 avg-zval = 34.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930824 bytes = 3374256 } seq expand size = 6515768 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5122 value = 3.3707 dict = 1.00 all = 3.5029 } Zip/UnZip{ index = 0.0951 value = 0.2967 dict = 1.00 all = 0.2855 } ---------------------------- total value len = 0.560567 GB avg = 0.117 KB (by entry num) total key len = 0.134405 GB avg = 0.028 KB total ukey len = 0.096003 GB avg = 0.020 KB total ukey num = 0.004800165 Billion total entry num = 0.004800165 Billion write speed all = 0.009240087 MB/sec (with seq num) write speed all = 0.008729517 MB/sec (without seq num) 2024/11/23-05:45:55.776845 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247207: 930824 keys, 38316258 bytes, temperature: kWarm 2024/11/23-05:45:55.776908 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311955776875, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247207, "file_size": 38316258, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3401489949, "largest_seqno": 3817786641, "table_properties": {"data_size": 32086368, "index_size": 1770944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26063072, "raw_average_key_size": 28, "raw_value_size": 108154680, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930824, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311952, "oldest_key_time": 0, "file_creation_time": 1732311952, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247207, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:45:55.818939 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247208.sst) /dev/shm/ToplingTemp/Topling-425412-UiojCQ 2024/11/23-05:45:56.784346 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247208.sst): first pass time = 0.97's, 139.032'MB/sec 2024/11/23-05:45:58.585604 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247208.sst): old prealloc_size = 147639500, real_size = 32478000 2024/11/23-05:45:58.586175 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247208.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921682 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.62 avg-zval = 33.07 2024/11/23-05:45:58.586991 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247208.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:45:59.478004 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247208.sst): second pass time = 1.80's, 60.204'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.231'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.550'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.302'MB/sec write SST data time = 0.13's, 285.340'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32812.101'MB/sec zip my value time = 1.80's, unzip length = 0.108'GB zip my value throughput = 60.204'MB/sec zip pipeline throughput = 133.303'MB/sec entries = 921682 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.62 avg-zval = 33.07 usrkeys = 921682 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.62 avg-zval = 33.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921682 bytes = 3456320 } seq expand size = 6451774 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4444 value = 3.5565 dict = 1.00 all = 3.6479 } Zip/UnZip{ index = 0.0957 value = 0.2812 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 0.675429 GB avg = 0.118 KB (by entry num) total key len = 0.160212 GB avg = 0.028 KB total ukey len = 0.114437 GB avg = 0.020 KB total ukey num = 0.005721847 Billion total entry num = 0.005721847 Billion write speed all = 0.011109827 MB/sec (with seq num) write speed all = 0.010501252 MB/sec (without seq num) 2024/11/23-05:45:59.510482 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247208: 921682 keys, 36793330 bytes, temperature: kWarm 2024/11/23-05:45:59.510546 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311959510513, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247208, "file_size": 36793330, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3817785034, "table_properties": {"data_size": 30482464, "index_size": 1764928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25807096, "raw_average_key_size": 28, "raw_value_size": 108410680, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921682, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311955, "oldest_key_time": 0, "file_creation_time": 1732311955, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247208, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:45:59.550772 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247209.sst) /dev/shm/ToplingTemp/Topling-425412-tRCaAS 2024/11/23-05:46:00.352268 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247209.sst): first pass time = 0.80's, 167.467'MB/sec 2024/11/23-05:46:01.899840 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247209.sst): old prealloc_size = 147639500, real_size = 33285792 2024/11/23-05:46:01.900566 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247209.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928486 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.56 avg-zval = 33.68 2024/11/23-05:46:01.901391 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247209.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:46:03.000400 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247209.sst): second pass time = 1.55's, 70.013'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 33.973'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2530.140'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.246'MB/sec write SST data time = 0.16's, 237.196'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30963.545'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 70.013'MB/sec zip pipeline throughput = 70.013'MB/sec entries = 928486 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.56 avg-zval = 33.68 usrkeys = 928486 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.56 avg-zval = 33.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928486 bytes = 3249712 } seq expand size = 6499402 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4439 value = 3.4602 dict = 1.00 all = 3.5898 } Zip/UnZip{ index = 0.0957 value = 0.2890 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 0.790149 GB avg = 0.119 KB (by entry num) total key len = 0.186209 GB avg = 0.028 KB total ukey len = 0.133007 GB avg = 0.020 KB total ukey num = 0.006650333 Billion total entry num = 0.006650333 Billion write speed all = 0.012980050 MB/sec (with seq num) write speed all = 0.012272755 MB/sec (without seq num) 2024/11/23-05:46:03.033332 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247209: 928486 keys, 37388680 bytes, temperature: kWarm 2024/11/23-05:46:03.033407 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311963033374, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247209, "file_size": 37388680, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3817786656, "table_properties": {"data_size": 31275408, "index_size": 1778048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25997608, "raw_average_key_size": 28, "raw_value_size": 108220176, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928486, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311959, "oldest_key_time": 0, "file_creation_time": 1732311959, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247209, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:46:03.072236 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247210.sst) /dev/shm/ToplingTemp/Topling-425412-k7BhRQ 2024/11/23-05:46:04.002088 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247210.sst): first pass time = 0.93's, 144.349'MB/sec 2024/11/23-05:46:05.811456 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247210.sst): old prealloc_size = 147639500, real_size = 33159792 2024/11/23-05:46:05.812002 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247210.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928288 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.59 avg-zval = 33.57 2024/11/23-05:46:05.812793 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247210.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:46:06.855697 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247210.sst): second pass time = 1.81's, 59.837'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 35.795'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3386.709'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.254'MB/sec write SST data time = 0.13's, 287.296'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33924.781'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.837'MB/sec zip pipeline throughput = 120.906'MB/sec entries = 928288 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.59 avg-zval = 33.57 usrkeys = 928288 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.59 avg-zval = 33.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928288 bytes = 3365056 } seq expand size = 6498016 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5132 value = 3.4731 dict = 1.00 all = 3.5908 } Zip/UnZip{ index = 0.0951 value = 0.2879 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 0.904873 GB avg = 0.119 KB (by entry num) total key len = 0.212201 GB avg = 0.028 KB total ukey len = 0.151572 GB avg = 0.020 KB total ukey num = 0.007578621 Billion total entry num = 0.007578621 Billion write speed all = 0.014850015 MB/sec (with seq num) write speed all = 0.014044034 MB/sec (without seq num) 2024/11/23-05:46:06.896036 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247210: 928288 keys, 37378364 bytes, temperature: kWarm 2024/11/23-05:46:06.896100 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311966896067, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247210, "file_size": 37378364, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3817786818, "table_properties": {"data_size": 31161568, "index_size": 1765952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25992064, "raw_average_key_size": 28, "raw_value_size": 108225760, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928288, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311963, "oldest_key_time": 0, "file_creation_time": 1732311963, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247210, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:46:06.932965 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247211.sst) /dev/shm/ToplingTemp/Topling-425412-OtChBS 2024/11/23-05:46:07.825961 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247211.sst): first pass time = 0.89's, 150.306'MB/sec 2024/11/23-05:46:09.222873 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247211.sst): old prealloc_size = 147639500, real_size = 32889600 2024/11/23-05:46:09.223596 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247211.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928360 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.58 avg-zval = 33.26 2024/11/23-05:46:09.224466 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247211.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:46:10.243349 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247211.sst): second pass time = 1.40's, 77.545'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 36.364'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2524.657'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.270'MB/sec write SST data time = 0.16's, 231.794'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33879.824'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.545'MB/sec zip pipeline throughput = 77.545'MB/sec entries = 928360 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.58 avg-zval = 33.26 usrkeys = 928360 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.58 avg-zval = 33.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928360 bytes = 3365312 } seq expand size = 6498520 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4425 value = 3.5047 dict = 1.00 all = 3.6169 } Zip/UnZip{ index = 0.0958 value = 0.2853 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 1.019595 GB avg = 0.120 KB (by entry num) total key len = 0.238195 GB avg = 0.028 KB total ukey len = 0.170140 GB avg = 0.020 KB total ukey num = 0.008506981 Billion total entry num = 0.008506981 Billion write speed all = 0.016719900 MB/sec (with seq num) write speed all = 0.015815228 MB/sec (without seq num) 2024/11/23-05:46:10.279069 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247211: 928360 keys, 37108502 bytes, temperature: kWarm 2024/11/23-05:46:10.279131 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311970279100, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247211, "file_size": 37108502, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3817786704, "table_properties": {"data_size": 30879280, "index_size": 1778048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25994080, "raw_average_key_size": 28, "raw_value_size": 108223768, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928360, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311966, "oldest_key_time": 0, "file_creation_time": 1732311966, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247211, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:46:10.318465 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247212.sst) /dev/shm/ToplingTemp/Topling-425412-SOMVPR 2024/11/23-05:46:11.211166 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247212.sst): first pass time = 0.89's, 150.356'MB/sec 2024/11/23-05:46:12.550765 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247212.sst): old prealloc_size = 147639500, real_size = 32079472 2024/11/23-05:46:12.551360 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247212.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930180 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0301 GB avg-val = 116.29 avg-zval = 32.33 2024/11/23-05:46:12.552120 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247212.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:46:13.470305 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247212.sst): second pass time = 1.34's, 80.821'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 39.354'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3140.555'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.294'MB/sec write SST data time = 0.13's, 280.334'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31947.933'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.821'MB/sec zip pipeline throughput = 80.821'MB/sec entries = 930180 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.29 avg-zval = 32.33 usrkeys = 930180 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.29 avg-zval = 32.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930180 bytes = 3255648 } seq expand size = 6511260 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5121 value = 3.5965 dict = 1.00 all = 3.7090 } Zip/UnZip{ index = 0.0951 value = 0.2780 dict = 1.00 all = 0.2696 } ---------------------------- total value len = 1.134279 GB avg = 0.120 KB (by entry num) total key len = 0.264241 GB avg = 0.028 KB total ukey len = 0.188743 GB avg = 0.020 KB total ukey num = 0.009437161 Billion total entry num = 0.009437161 Billion write speed all = 0.018589824 MB/sec (with seq num) write speed all = 0.017586276 MB/sec (without seq num) 2024/11/23-05:46:13.505166 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247212: 930180 keys, 36187122 bytes, temperature: kWarm 2024/11/23-05:46:13.505238 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311973505197, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247212, "file_size": 36187122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3817786715, "table_properties": {"data_size": 30077024, "index_size": 1769728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26045040, "raw_average_key_size": 28, "raw_value_size": 108172744, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930180, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311970, "oldest_key_time": 0, "file_creation_time": 1732311970, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247212, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:46:13.541125 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247213.sst) /dev/shm/ToplingTemp/Topling-425412-cUGp5P 2024/11/23-05:46:14.400924 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247213.sst): first pass time = 0.86's, 156.110'MB/sec 2024/11/23-05:46:15.721975 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247213.sst): old prealloc_size = 147639500, real_size = 31518352 2024/11/23-05:46:15.722522 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247213.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927947 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0295 GB avg-val = 116.64 avg-zval = 31.81 2024/11/23-05:46:15.723322 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247213.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:46:16.595992 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247213.sst): second pass time = 1.32's, 82.014'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 40.888'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3366.809'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 275.213'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31854.990'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.014'MB/sec zip pipeline throughput = 82.014'MB/sec entries = 927947 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.64 avg-zval = 31.81 usrkeys = 927947 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.64 avg-zval = 31.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927947 bytes = 3363824 } seq expand size = 6495629 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5131 value = 3.6664 dict = 1.00 all = 3.7558 } Zip/UnZip{ index = 0.0951 value = 0.2727 dict = 1.00 all = 0.2663 } ---------------------------- total value len = 1.249010 GB avg = 0.121 KB (by entry num) total key len = 0.290223 GB avg = 0.028 KB total ukey len = 0.207302 GB avg = 0.020 KB total ukey num = 0.010365108 Billion total entry num = 0.010365108 Billion write speed all = 0.020459408 MB/sec (with seq num) write speed all = 0.019357228 MB/sec (without seq num) 2024/11/23-05:46:16.629147 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247213: 927947 keys, 35735932 bytes, temperature: kWarm 2024/11/23-05:46:16.629226 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311976629187, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247213, "file_size": 35735932, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3817786730, "table_properties": {"data_size": 29520896, "index_size": 1765312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25982516, "raw_average_key_size": 28, "raw_value_size": 108235330, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927947, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311973, "oldest_key_time": 0, "file_creation_time": 1732311973, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247213, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:46:16.668794 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247214.sst) /dev/shm/ToplingTemp/Topling-425412-T4EboQ 2024/11/23-05:46:17.441960 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247214.sst): first pass time = 0.77's, 173.602'MB/sec 2024/11/23-05:46:18.769462 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247214.sst): old prealloc_size = 147639500, real_size = 32104160 2024/11/23-05:46:18.770000 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247214.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925704 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0301 GB avg-val = 116.99 avg-zval = 32.53 2024/11/23-05:46:18.770809 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247214.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:46:19.662625 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247214.sst): second pass time = 1.33's, 81.626'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.659'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3403.625'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.303'MB/sec write SST data time = 0.13's, 281.144'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30792.247'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.626'MB/sec zip pipeline throughput = 81.626'MB/sec entries = 925704 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.99 avg-zval = 32.53 usrkeys = 925704 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.99 avg-zval = 32.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925704 bytes = 3355696 } seq expand size = 6479928 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5140 value = 3.5966 dict = 1.00 all = 3.6960 } Zip/UnZip{ index = 0.0951 value = 0.2780 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 1.363788 GB avg = 0.121 KB (by entry num) total key len = 0.316143 GB avg = 0.028 KB total ukey len = 0.225816 GB avg = 0.020 KB total ukey num = 0.011290812 Billion total entry num = 0.011290812 Billion write speed all = 0.022328646 MB/sec (with seq num) write speed all = 0.021128079 MB/sec (without seq num) 2024/11/23-05:46:19.694608 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247214: 925704 keys, 36314538 bytes, temperature: kWarm 2024/11/23-05:46:19.694672 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311979694639, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247214, "file_size": 36314538, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3817786745, "table_properties": {"data_size": 30111632, "index_size": 1760896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919712, "raw_average_key_size": 28, "raw_value_size": 108298136, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925704, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311976, "oldest_key_time": 0, "file_creation_time": 1732311976, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247214, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:46:19.729514 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247215.sst) /dev/shm/ToplingTemp/Topling-425412-XOTiVS 2024/11/23-05:46:20.582752 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247215.sst): first pass time = 0.85's, 157.311'MB/sec 2024/11/23-05:46:21.945336 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247215.sst): old prealloc_size = 147639500, real_size = 34841408 2024/11/23-05:46:21.945937 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247215.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926508 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0328 GB avg-val = 116.86 avg-zval = 35.43 2024/11/23-05:46:21.947021 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247215.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:46:22.834899 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247215.sst): second pass time = 1.36's, 79.506'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.430'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3077.331'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.13's, 295.185'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33032.359'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.506'MB/sec zip pipeline throughput = 79.506'MB/sec entries = 926508 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.86 avg-zval = 35.43 usrkeys = 926508 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.48 avg-zval = 35.43 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 743511 bytes = 2788176 } seq expand size = 5204577 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.3776 value = 3.2987 dict = 1.00 all = 3.4742 } Zip/UnZip{ index = 0.0964 value = 0.3032 dict = 1.00 all = 0.2878 } ---------------------------- total value len = 1.477268 GB avg = 0.121 KB (by entry num) total key len = 0.342085 GB avg = 0.028 KB total ukey len = 0.244346 GB avg = 0.020 KB total ukey num = 0.012217320 Billion total entry num = 0.012217320 Billion write speed all = 0.024180746 MB/sec (with seq num) write speed all = 0.022881717 MB/sec (without seq num) 2024/11/23-05:46:22.869213 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247215: 926508 keys, 38632554 bytes, temperature: kWarm 2024/11/23-05:46:22.869279 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311982869246, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247215, "file_size": 38632554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3817784488, "table_properties": {"data_size": 32823984, "index_size": 1785600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942224, "raw_average_key_size": 28, "raw_value_size": 108275532, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926508, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311979, "oldest_key_time": 0, "file_creation_time": 1732311979, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247215, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:46:22.904124 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247216.sst) /dev/shm/ToplingTemp/Topling-425412-r9bJZQ 2024/11/23-05:46:23.811551 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247216.sst): first pass time = 0.91's, 147.916'MB/sec 2024/11/23-05:46:25.153302 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247216.sst): old prealloc_size = 147639500, real_size = 33878880 2024/11/23-05:46:25.153886 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247216.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933840 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0319 GB avg-val = 115.73 avg-zval = 34.13 2024/11/23-05:46:25.154710 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247216.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-05:46:26.065976 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247216.sst): second pass time = 1.34's, 80.584'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.762'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3158.767'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 295.678'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32415.707'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.584'MB/sec zip pipeline throughput = 80.584'MB/sec entries = 933840 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.73 avg-zval = 34.13 usrkeys = 933840 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.73 avg-zval = 34.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933840 bytes = 3385184 } seq expand size = 6536880 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5143 value = 3.3911 dict = 1.00 all = 3.5214 } Zip/UnZip{ index = 0.0951 value = 0.2949 dict = 1.00 all = 0.2840 } ---------------------------- total value len = 1.591875 GB avg = 0.121 KB (by entry num) total key len = 0.368232 GB avg = 0.028 KB total ukey len = 0.263023 GB avg = 0.020 KB total ukey num = 0.013151160 Billion total entry num = 0.013151160 Billion write speed all = 0.026050376 MB/sec (with seq num) write speed all = 0.024652115 MB/sec (without seq num) 2024/11/23-05:46:26.098386 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247216: 933840 keys, 38114610 bytes, temperature: kWarm 2024/11/23-05:46:26.098449 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311986098415, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247216, "file_size": 38114610, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3817786967, "table_properties": {"data_size": 31868944, "index_size": 1776320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26147520, "raw_average_key_size": 28, "raw_value_size": 108070260, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933840, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311982, "oldest_key_time": 0, "file_creation_time": 1732311982, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247216, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:46:26.134363 140024793790016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247217.sst) /dev/shm/ToplingTemp/Topling-425412-XW6J2P 2024/11/23-05:46:26.324787 140024793790016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247217.sst): first pass time = 0.19's, 248.949'MB/sec 2024/11/23-05:46:26.789313 140024793790016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247217.sst): old prealloc_size = 147639500, real_size = 12781168 2024/11/23-05:46:26.789551 140024793790016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247217.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 324630 prefix = 4 raw-key = 0.0052 GB zip-key = 0.0006 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0383 GB zip-val = 0.0121 GB avg-val = 118.00 avg-zval = 37.20 2024/11/23-05:46:26.789848 140024793790016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247217.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0008 GB, waited 0.000 sec, Key+Value = 0.047 GB 2024/11/23-05:46:27.116393 140024793790016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00/247217.sst): second pass time = 0.46's, 82.572'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.33's, 43.262'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2931.696'MB/sec (index lex order gen) rebuild zvType time = 0.28's, 0.289'MB/sec write SST data time = 0.05's, 310.618'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.392'MB, throughput = 12099.485'MB/sec zip my value time = 0.46's, unzip length = 0.038'GB zip my value throughput = 82.572'MB/sec zip pipeline throughput = 82.572'MB/sec entries = 324630 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 37.20 usrkeys = 324630 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 37.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 324630 bytes = 1055056 } seq expand size = 2272410 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0065 GB value = 0.0383 GB dict = 0.39 MB all = 0.0474 GB } __ZipSize{ index = 0.0006 GB value = 0.0121 GB dict = 0.39 MB all = 0.0141 GB } UnZip/Zip{ index = 10.4294 value = 3.1718 dict = 1.00 all = 3.3496 } Zip/UnZip{ index = 0.0959 value = 0.3153 dict = 1.00 all = 0.2985 } ---------------------------- total value len = 1.632454 GB avg = 0.121 KB (by entry num) total key len = 0.377322 GB avg = 0.028 KB total ukey len = 0.269516 GB avg = 0.020 KB total ukey num = 0.013475790 Billion total entry num = 0.013475790 Billion write speed all = 0.026710110 MB/sec (with seq num) write speed all = 0.025277354 MB/sec (without seq num) 2024/11/23-05:46:27.132443 140024793790016 [compaction_job.cc:1948] [default] [JOB 20439] Generated table #247217: 324630 keys, 14149629 bytes, temperature: kWarm 2024/11/23-05:46:27.132504 140024793790016 EVENT_LOG_v1 {"time_micros": 1732311987132473, "cf_name": "default", "job": 20439, "event": "table_file_creation", "file_number": 247217, "file_size": 14149629, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3751296070, "largest_seqno": 3814014060, "table_properties": {"data_size": 12077280, "index_size": 622528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 9089640, "raw_average_key_size": 28, "raw_value_size": 38306340, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 324630, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732311986, "oldest_key_time": 0, "file_creation_time": 1732311986, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 247217, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-05:46:27.147319 140024793790016 [compaction_job.cc:703] job-20439: subcompact[0], size: 0.533762 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-20439/att-00 [247203,247204,247205,247206,247207,247208,247209,247210,247211,247212,247213,247214,247215,247216,247217] 2024/11/23-05:46:27.241429 140024793790016 (Original Log Time 2024/11/23-05:45:37.799954) EVENT_LOG_v1 {"time_micros": 1732311937799916, "job": 20439, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [247164], "files_L3": [247199, 247200, 247104, 247105, 247106, 247107, 247108, 247109, 247110, 247111, 247112, 247113, 247114, 247115, 247178], "score": 0.99792, "input_data_size": 552504470, "oldest_snapshot_seqno": 3818200763} 2024/11/23-05:46:27.241432 140024793790016 (Original Log Time 2024/11/23-05:46:27.240662) [compaction_job.cc:2022] [default] [JOB 20439] Compacted 1@2 + 15@3 files to L3 => 533762280 bytes 2024/11/23-05:46:27.241434 140024793790016 (Original Log Time 2024/11/23-05:46:27.241364) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.2 rd, 10.8 wr, level 3, files in(1, 15) out(15 +0 blob) MB in(27.1, 499.8 +0.0 blob) out(509.0 +0.0 blob), read-write-amplify(38.2) write-amplify(18.8) OK, records in: 13557357, records dropped: 81567 output_compression: Snappy 2024/11/23-05:46:27.241440 140024793790016 (Original Log Time 2024/11/23-05:46:27.241409) EVENT_LOG_v1 {"time_micros": 1732311987241388, "cf": "default", "job": 20439, "event": "compaction_finished", "compaction_time_micros": 49347371, "compaction_time_cpu_micros": 21970453, "output_level": 3, "num_output_files": 15, "total_output_size": 533762280, "num_input_records": 13557357, "num_output_records": 13475790, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 348676949, "file_range_sync_nanos": 0, "file_fsync_nanos": 13485, "file_prepare_write_nanos": 34687, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-05:46:27.374032 140024793790016 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732311937,"executesMs":49559,"compactionJobId":20439,"attempt":0,"compactionInputRawBytes":1938027811,"compactionInputZipBytes":552504470,"compactionOutputRawBytes":0,"compactionOutputZipBytes":533762280,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-f3d3427194d9e2fa49a485afc6338900-ab2cc08f3ffcad36-00"}}, response = 2024/11/23-05:46:27.374112 140024793790016 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-20439/att-00: olev 3, work 49.559 s, result 4.952 ms, install 0.795 ms, input{raw 1.805 GiB zip 526.909 MiB}