2024/11/24-03:23:28.708144 139957525694016 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00: fork to child time = 0.008311 sec 2024/11/24-03:23:28.708887 139957525694016 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00 2024/11/24-03:23:28.714681 139957525694016 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00 2024/11/24-03:23:28.715575 139957525694016 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-03:23:28.715818 139957525694016 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-03:23:28.765525 139957525694016 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40772/att-00/MANIFEST-206347 2024/11/24-03:23:28.797661 139957525694016 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40772/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 290242, last_sequence is 4415989328, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 290077 2024/11/24-03:23:28.797668 139957525694016 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-03:23:28.797801 139957525694016 [version_set.cc:5721] Creating manifest 290242 2024/11/24-03:23:28.833510 139957525694016 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-03:23:28.852058 139957525694016 [compaction_job.cc:2437] [default] [JOB 40772] Compacting 1@2 + 20@3 files to L3, score 0.70, subcompactions 1 : 1 2024/11/24-03:23:28.852072 139957525694016 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 30534 Base level 2, inputs: [290192(27MB)], [290225(35MB) 290226(1535KB) 290161(33MB) 290162(33MB) 290163(35MB) 290164(35MB) 290165(35MB) 290166(31MB) 290167(33MB) 290168(7203KB) 290133(32MB) 290134(33MB) 290135(33MB) 290136(34MB) 290137(33MB) 290138(35MB) 290139(33MB) 290140(35MB) 290141(32MB) 290233(33MB)] 2024/11/24-03:23:28.859139 139957525694016 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:23:28.859167 139957525694016 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:23:28.859186 139957525694016 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-03:23:28.859192 139957525694016 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:23:28.859203 139957525694016 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-03:23:29.595532 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290243.sst) /dev/shm/ToplingTemp/Topling-257932-ul5bNN 2024/11/24-03:23:30.143343 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290243.sst): first pass time = 0.55's, 245.026'MB/sec 2024/11/24-03:23:31.601354 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290243.sst): old prealloc_size = 147639500, real_size = 32925376 2024/11/24-03:23:31.601885 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290243.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926947 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.80 avg-zval = 33.37 2024/11/24-03:23:31.602700 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290243.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:23:32.447389 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290243.sst): second pass time = 1.46's, 74.299'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.763'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3454.891'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 292.771'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35113.690'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.299'MB/sec zip pipeline throughput = 74.299'MB/sec entries = 926947 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 33.37 usrkeys = 926947 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 33.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926947 bytes = 3244336 } seq expand size = 6488629 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5128 value = 3.5003 dict = 1.00 all = 3.6251 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 2251.440325 GB avg = 0.200 KB (by entry num) total key len = 306.908917 GB avg = 0.027 KB total ukey len = 237.373266 GB avg = 0.015 KB total ukey num = 15.739588110 Billion total entry num = 11.280902058 Billion write speed all = 38.471282535 MB/sec (with seq num) write speed all = 37.114186285 MB/sec (without seq num) 2024/11/24-03:23:32.494034 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290243: 926947 keys, 37024138 bytes, temperature: kWarm 2024/11/24-03:23:32.494134 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389812494071, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290243, "file_size": 37024138, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4415795611, "table_properties": {"data_size": 30929968, "index_size": 1763456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954516, "raw_average_key_size": 28, "raw_value_size": 108263342, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926947, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389809, "oldest_key_time": 0, "file_creation_time": 1732389809, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290243, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:23:32.532548 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290244.sst) /dev/shm/ToplingTemp/Topling-257932-ho5mFP 2024/11/24-03:23:33.557710 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290244.sst): first pass time = 1.03's, 130.928'MB/sec 2024/11/24-03:23:34.976722 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290244.sst): old prealloc_size = 147639500, real_size = 30755264 2024/11/24-03:23:34.977286 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290244.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926012 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0288 GB avg-val = 116.94 avg-zval = 31.05 2024/11/24-03:23:34.978055 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290244.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:23:35.802664 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290244.sst): second pass time = 1.42's, 76.393'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.055'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3248.129'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 273.850'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34719.954'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.393'MB/sec zip pipeline throughput = 76.393'MB/sec entries = 926012 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.94 avg-zval = 31.05 usrkeys = 926012 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.94 avg-zval = 31.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926012 bytes = 3125312 } seq expand size = 6482084 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.4454 value = 3.7665 dict = 1.00 all = 3.8642 } Zip/UnZip{ index = 0.0957 value = 0.2655 dict = 1.00 all = 0.2588 } ---------------------------- total value len = 2251.555097 GB avg = 0.200 KB (by entry num) total key len = 306.934845 GB avg = 0.027 KB total ukey len = 237.391787 GB avg = 0.015 KB total ukey num = 15.740514122 Billion total entry num = 11.281828070 Billion write speed all = 38.471457236 MB/sec (with seq num) write speed all = 37.114318061 MB/sec (without seq num) 2024/11/24-03:23:35.847294 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290244: 926012 keys, 34733800 bytes, temperature: kWarm 2024/11/24-03:23:35.847361 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389815847326, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290244, "file_size": 34733800, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4303151998, "largest_seqno": 4415795437, "table_properties": {"data_size": 28750512, "index_size": 1773056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928336, "raw_average_key_size": 28, "raw_value_size": 108289440, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926012, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389812, "oldest_key_time": 0, "file_creation_time": 1732389812, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290244, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:23:35.888231 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290245.sst) /dev/shm/ToplingTemp/Topling-257932-b9mCVN 2024/11/24-03:23:36.834791 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290245.sst): first pass time = 0.95's, 141.801'MB/sec 2024/11/24-03:23:38.269024 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290245.sst): old prealloc_size = 147639500, real_size = 31719488 2024/11/24-03:23:38.269542 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290245.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919903 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0297 GB avg-val = 117.90 avg-zval = 32.33 2024/11/24-03:23:38.271410 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290245.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:23:39.085655 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290245.sst): second pass time = 1.43's, 75.667'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.840'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3521.337'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 284.100'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 37664.271'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.667'MB/sec zip pipeline throughput = 75.667'MB/sec entries = 919903 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.90 avg-zval = 32.33 usrkeys = 919903 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.90 avg-zval = 32.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919903 bytes = 3219680 } seq expand size = 6439321 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5127 value = 3.6471 dict = 1.00 all = 3.7491 } Zip/UnZip{ index = 0.0951 value = 0.2742 dict = 1.00 all = 0.2667 } ---------------------------- total value len = 2251.669997 GB avg = 0.200 KB (by entry num) total key len = 306.960603 GB avg = 0.027 KB total ukey len = 237.410185 GB avg = 0.015 KB total ukey num = 15.741434025 Billion total entry num = 11.282747973 Billion write speed all = 38.471673087 MB/sec (with seq num) write speed all = 37.114490252 MB/sec (without seq num) 2024/11/24-03:23:39.131688 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290245: 919903 keys, 35799578 bytes, temperature: kWarm 2024/11/24-03:23:39.131757 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389819131722, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290245, "file_size": 35799578, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4415793802, "table_properties": {"data_size": 29739248, "index_size": 1750080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25757284, "raw_average_key_size": 28, "raw_value_size": 108460534, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919903, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389815, "oldest_key_time": 0, "file_creation_time": 1732389815, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290245, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:23:39.187771 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290246.sst) /dev/shm/ToplingTemp/Topling-257932-9dXESN 2024/11/24-03:23:40.098129 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290246.sst): first pass time = 0.91's, 147.440'MB/sec 2024/11/24-03:23:41.587828 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290246.sst): old prealloc_size = 147639500, real_size = 33123424 2024/11/24-03:23:41.588421 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290246.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926680 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.84 avg-zval = 33.58 2024/11/24-03:23:41.589217 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290246.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:23:42.471278 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290246.sst): second pass time = 1.49's, 72.718'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.133'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3114.793'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 281.816'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35124.053'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.718'MB/sec zip pipeline throughput = 72.718'MB/sec entries = 926680 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.84 avg-zval = 33.58 usrkeys = 926680 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.84 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926680 bytes = 3243392 } seq expand size = 6486760 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4427 value = 3.4795 dict = 1.00 all = 3.6060 } Zip/UnZip{ index = 0.0958 value = 0.2874 dict = 1.00 all = 0.2773 } ---------------------------- total value len = 2251.784754 GB avg = 0.200 KB (by entry num) total key len = 306.986550 GB avg = 0.027 KB total ukey len = 237.428718 GB avg = 0.015 KB total ukey num = 15.742360705 Billion total entry num = 11.283674653 Billion write speed all = 38.471830266 MB/sec (with seq num) write speed all = 37.114605053 MB/sec (without seq num) 2024/11/24-03:23:42.520186 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290246: 926680 keys, 37221178 bytes, temperature: kWarm 2024/11/24-03:23:42.520255 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389822520220, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290246, "file_size": 37221178, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4415796068, "table_properties": {"data_size": 31116816, "index_size": 1774784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947040, "raw_average_key_size": 28, "raw_value_size": 108270804, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926680, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389819, "oldest_key_time": 0, "file_creation_time": 1732389819, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290246, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:23:42.570533 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290247.sst) /dev/shm/ToplingTemp/Topling-257932-G1MmlP 2024/11/24-03:23:43.619393 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290247.sst): first pass time = 1.05's, 127.970'MB/sec 2024/11/24-03:23:45.123421 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290247.sst): old prealloc_size = 147639500, real_size = 33479424 2024/11/24-03:23:45.124019 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290247.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926449 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.87 avg-zval = 33.97 2024/11/24-03:23:45.124840 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290247.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:23:45.998620 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290247.sst): second pass time = 1.50's, 72.029'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.936'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3070.334'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.310'MB/sec write SST data time = 0.13's, 294.243'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36486.510'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.029'MB/sec zip pipeline throughput = 72.029'MB/sec entries = 926449 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.87 avg-zval = 33.97 usrkeys = 926449 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.87 avg-zval = 33.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926449 bytes = 3242592 } seq expand size = 6485143 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4480 value = 3.4402 dict = 1.00 all = 3.5718 } Zip/UnZip{ index = 0.0957 value = 0.2907 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 2251.899517 GB avg = 0.200 KB (by entry num) total key len = 307.012490 GB avg = 0.027 KB total ukey len = 237.447247 GB avg = 0.015 KB total ukey num = 15.743287154 Billion total entry num = 11.284601102 Billion write speed all = 38.471905432 MB/sec (with seq num) write speed all = 37.114640766 MB/sec (without seq num) 2024/11/24-03:23:46.047960 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290247: 926449 keys, 37576692 bytes, temperature: kWarm 2024/11/24-03:23:46.048062 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389826048016, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290247, "file_size": 37576692, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4415796248, "table_properties": {"data_size": 31474160, "index_size": 1773440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25940572, "raw_average_key_size": 28, "raw_value_size": 108277286, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926449, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389822, "oldest_key_time": 0, "file_creation_time": 1732389822, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290247, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:23:46.085501 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290248.sst) /dev/shm/ToplingTemp/Topling-257932-Q9wmcM 2024/11/24-03:23:47.122752 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290248.sst): first pass time = 1.04's, 129.402'MB/sec 2024/11/24-03:23:48.606386 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290248.sst): old prealloc_size = 147639500, real_size = 32698592 2024/11/24-03:23:48.607059 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290248.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926772 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 116.82 avg-zval = 33.12 2024/11/24-03:23:48.607900 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290248.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:23:49.470316 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290248.sst): second pass time = 1.48's, 73.022'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.597'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2685.799'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.15's, 248.854'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27822.855'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.022'MB/sec zip pipeline throughput = 73.022'MB/sec entries = 926772 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.82 avg-zval = 33.12 usrkeys = 926772 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.82 avg-zval = 33.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926772 bytes = 3243712 } seq expand size = 6487404 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4438 value = 3.5276 dict = 1.00 all = 3.6472 } Zip/UnZip{ index = 0.0958 value = 0.2835 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 2252.014272 GB avg = 0.200 KB (by entry num) total key len = 307.038440 GB avg = 0.027 KB total ukey len = 237.465783 GB avg = 0.015 KB total ukey num = 15.744213926 Billion total entry num = 11.285527874 Billion write speed all = 38.472012811 MB/sec (with seq num) write speed all = 37.114707521 MB/sec (without seq num) 2024/11/24-03:23:49.516437 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290248: 926772 keys, 36800358 bytes, temperature: kWarm 2024/11/24-03:23:49.516506 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389829516472, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290248, "file_size": 36800358, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4415795480, "table_properties": {"data_size": 30691920, "index_size": 1774784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949616, "raw_average_key_size": 28, "raw_value_size": 108268208, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926772, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389826, "oldest_key_time": 0, "file_creation_time": 1732389826, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290248, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:23:49.552764 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290249.sst) /dev/shm/ToplingTemp/Topling-257932-SJ9TlM 2024/11/24-03:23:50.376496 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290249.sst): first pass time = 0.82's, 162.946'MB/sec 2024/11/24-03:23:51.802489 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290249.sst): old prealloc_size = 147639500, real_size = 29805168 2024/11/24-03:23:51.803107 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290249.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934105 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0278 GB avg-val = 115.69 avg-zval = 29.74 2024/11/24-03:23:51.803885 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290249.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:23:52.642784 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290249.sst): second pass time = 1.42's, 75.841'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 40.373'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3006.921'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 265.976'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35368.000'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 75.841'MB/sec zip pipeline throughput = 75.841'MB/sec entries = 934105 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.69 avg-zval = 29.74 usrkeys = 934105 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.69 avg-zval = 29.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934105 bytes = 3269376 } seq expand size = 6538735 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0278 GB dict = 1.08 MB all = 0.0339 GB } UnZip/Zip{ index = 10.4443 value = 3.8896 dict = 1.00 all = 3.9564 } Zip/UnZip{ index = 0.0957 value = 0.2571 dict = 1.00 all = 0.2528 } ---------------------------- total value len = 2252.128874 GB avg = 0.200 KB (by entry num) total key len = 307.064595 GB avg = 0.027 KB total ukey len = 237.484465 GB avg = 0.015 KB total ukey num = 15.745148031 Billion total entry num = 11.286461979 Billion write speed all = 38.472294009 MB/sec (with seq num) write speed all = 37.114941112 MB/sec (without seq num) 2024/11/24-03:23:52.687516 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290249: 934105 keys, 33923796 bytes, temperature: kWarm 2024/11/24-03:23:52.687588 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389832687551, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290249, "file_size": 33923796, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4415794958, "table_properties": {"data_size": 27782752, "index_size": 1788736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26154940, "raw_average_key_size": 28, "raw_value_size": 108062846, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934105, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389829, "oldest_key_time": 0, "file_creation_time": 1732389829, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290249, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:23:52.724606 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290250.sst) /dev/shm/ToplingTemp/Topling-257932-FGTEBN 2024/11/24-03:23:53.731950 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290250.sst): first pass time = 1.01's, 133.244'MB/sec 2024/11/24-03:23:55.143292 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290250.sst): old prealloc_size = 147639500, real_size = 29116080 2024/11/24-03:23:55.143824 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290250.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 943071 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0271 GB avg-val = 114.32 avg-zval = 28.72 2024/11/24-03:23:55.145778 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290250.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:23:55.987514 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290250.sst): second pass time = 1.41's, 76.469'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 39.400'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3520.512'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.329'MB/sec write SST data time = 0.13's, 260.034'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.075'MB, throughput = 38770.746'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.469'MB/sec zip pipeline throughput = 76.469'MB/sec entries = 943071 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.32 avg-zval = 28.72 usrkeys = 943071 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.32 avg-zval = 28.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 943071 bytes = 3300768 } seq expand size = 6601497 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0271 GB dict = 1.07 MB all = 0.0333 GB } UnZip/Zip{ index = 10.5126 value = 3.9804 dict = 1.00 all = 4.0352 } Zip/UnZip{ index = 0.0951 value = 0.2512 dict = 1.00 all = 0.2478 } ---------------------------- total value len = 2252.243287 GB avg = 0.200 KB (by entry num) total key len = 307.091001 GB avg = 0.027 KB total ukey len = 237.503326 GB avg = 0.015 KB total ukey num = 15.746091102 Billion total entry num = 11.287405050 Billion write speed all = 38.472476497 MB/sec (with seq num) write speed all = 37.115078434 MB/sec (without seq num) 2024/11/24-03:23:56.031180 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290250: 943071 keys, 33261594 bytes, temperature: kWarm 2024/11/24-03:23:56.031245 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389836031210, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290250, "file_size": 33261594, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4415796007, "table_properties": {"data_size": 27085984, "index_size": 1794176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26405988, "raw_average_key_size": 28, "raw_value_size": 107811750, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 943071, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389832, "oldest_key_time": 0, "file_creation_time": 1732389832, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290250, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:23:56.068159 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290251.sst) /dev/shm/ToplingTemp/Topling-257932-l8utSM 2024/11/24-03:23:57.160855 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290251.sst): first pass time = 1.09's, 122.836'MB/sec 2024/11/24-03:23:58.613970 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290251.sst): old prealloc_size = 147639500, real_size = 31462896 2024/11/24-03:23:58.614555 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290251.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920208 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.86 avg-zval = 32.01 2024/11/24-03:23:58.616538 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290251.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:23:59.434806 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290251.sst): second pass time = 1.45's, 74.680'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.301'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3224.510'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.331'MB/sec write SST data time = 0.13's, 281.757'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35982.876'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.680'MB/sec zip pipeline throughput = 74.680'MB/sec entries = 920208 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.86 avg-zval = 32.01 usrkeys = 920208 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.86 avg-zval = 32.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920208 bytes = 3220736 } seq expand size = 6441456 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3765 value = 3.6814 dict = 1.00 all = 3.7763 } Zip/UnZip{ index = 0.0964 value = 0.2716 dict = 1.00 all = 0.2648 } ---------------------------- total value len = 2252.358180 GB avg = 0.200 KB (by entry num) total key len = 307.116767 GB avg = 0.027 KB total ukey len = 237.521730 GB avg = 0.015 KB total ukey num = 15.747011310 Billion total entry num = 11.288325258 Billion write speed all = 38.472597247 MB/sec (with seq num) write speed all = 37.115158866 MB/sec (without seq num) 2024/11/24-03:23:59.479711 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290251: 920208 keys, 35541860 bytes, temperature: kWarm 2024/11/24-03:23:59.479784 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389839479749, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290251, "file_size": 35541860, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4276185595, "largest_seqno": 4415795742, "table_properties": {"data_size": 29459040, "index_size": 1773632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25765824, "raw_average_key_size": 28, "raw_value_size": 108451904, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920208, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389836, "oldest_key_time": 0, "file_creation_time": 1732389836, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290251, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:23:59.517191 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290252.sst) /dev/shm/ToplingTemp/Topling-257932-XvQ3bN 2024/11/24-03:24:00.364824 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290252.sst): first pass time = 0.85's, 158.351'MB/sec 2024/11/24-03:24:01.758510 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290252.sst): old prealloc_size = 147639500, real_size = 27786768 2024/11/24-03:24:01.759060 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290252.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.22 2024/11/24-03:24:01.759857 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290252.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:24:02.596120 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290252.sst): second pass time = 1.39's, 77.363'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 37.988'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3396.026'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.332'MB/sec write SST data time = 0.13's, 252.488'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 37961.028'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.363'MB/sec zip pipeline throughput = 77.363'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.00 avg-zval = 27.22 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.00 avg-zval = 27.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.3749 value = 4.1881 dict = 1.00 all = 4.2183 } Zip/UnZip{ index = 0.0964 value = 0.2388 dict = 1.00 all = 0.2371 } ---------------------------- total value len = 2252.472549 GB avg = 0.200 KB (by entry num) total key len = 307.143232 GB avg = 0.027 KB total ukey len = 237.540634 GB avg = 0.015 KB total ukey num = 15.747956506 Billion total entry num = 11.289270454 Billion write speed all = 38.472885996 MB/sec (with seq num) write speed all = 37.115398460 MB/sec (without seq num) 2024/11/24-03:24:02.638184 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290252: 945196 keys, 31818032 bytes, temperature: kWarm 2024/11/24-03:24:02.638255 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389842638220, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290252, "file_size": 31818032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4312248689, "largest_seqno": 4414214622, "table_properties": {"data_size": 25728192, "index_size": 1822080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389839, "oldest_key_time": 0, "file_creation_time": 1732389839, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290252, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:24:02.673562 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290253.sst) /dev/shm/ToplingTemp/Topling-257932-t4bAgP 2024/11/24-03:24:03.660744 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290253.sst): first pass time = 0.99's, 135.967'MB/sec 2024/11/24-03:24:05.171990 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290253.sst): old prealloc_size = 147639500, real_size = 31643536 2024/11/24-03:24:05.172529 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290253.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919689 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0297 GB avg-val = 117.94 avg-zval = 32.25 2024/11/24-03:24:05.174537 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290253.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:24:05.985943 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290253.sst): second pass time = 1.51's, 71.828'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.889'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3378.342'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 285.441'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32677.148'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.828'MB/sec zip pipeline throughput = 103.509'MB/sec entries = 919689 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.94 avg-zval = 32.25 usrkeys = 919689 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.94 avg-zval = 32.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919689 bytes = 3218928 } seq expand size = 6437823 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0297 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5149 value = 3.6565 dict = 1.00 all = 3.7572 } Zip/UnZip{ index = 0.0951 value = 0.2735 dict = 1.00 all = 0.2662 } ---------------------------- total value len = 2252.587453 GB avg = 0.200 KB (by entry num) total key len = 307.168983 GB avg = 0.027 KB total ukey len = 237.559028 GB avg = 0.015 KB total ukey num = 15.748876195 Billion total entry num = 11.290190143 Billion write speed all = 38.473039889 MB/sec (with seq num) write speed all = 37.115510932 MB/sec (without seq num) 2024/11/24-03:24:06.029936 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290253: 919689 keys, 35723202 bytes, temperature: kWarm 2024/11/24-03:24:06.030015 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389846029967, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290253, "file_size": 35723202, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4415795163, "table_properties": {"data_size": 29664128, "index_size": 1749312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25751292, "raw_average_key_size": 28, "raw_value_size": 108466442, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919689, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389842, "oldest_key_time": 0, "file_creation_time": 1732389842, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290253, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:24:06.066946 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290254.sst) /dev/shm/ToplingTemp/Topling-257932-DTs4PO 2024/11/24-03:24:07.087813 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290254.sst): first pass time = 1.02's, 131.482'MB/sec 2024/11/24-03:24:08.521177 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290254.sst): old prealloc_size = 147639500, real_size = 30729072 2024/11/24-03:24:08.521697 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290254.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934755 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0287 GB avg-val = 115.59 avg-zval = 30.72 2024/11/24-03:24:08.522524 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290254.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:24:09.369019 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290254.sst): second pass time = 1.43's, 75.423'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 41.108'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3584.962'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 270.745'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34724.614'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.423'MB/sec zip pipeline throughput = 75.423'MB/sec entries = 934755 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.59 avg-zval = 30.72 usrkeys = 934755 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.59 avg-zval = 30.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934755 bytes = 3271664 } seq expand size = 6543285 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.5117 value = 3.7624 dict = 1.00 all = 3.8512 } Zip/UnZip{ index = 0.0951 value = 0.2658 dict = 1.00 all = 0.2597 } ---------------------------- total value len = 2252.702041 GB avg = 0.200 KB (by entry num) total key len = 307.195157 GB avg = 0.027 KB total ukey len = 237.577723 GB avg = 0.015 KB total ukey num = 15.749810950 Billion total entry num = 11.291124898 Billion write speed all = 38.473199270 MB/sec (with seq num) write speed all = 37.115626947 MB/sec (without seq num) 2024/11/24-03:24:09.414023 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290254: 934755 keys, 34851178 bytes, temperature: kWarm 2024/11/24-03:24:09.414101 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389849414057, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290254, "file_size": 34851178, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4415795757, "table_properties": {"data_size": 28716704, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26173140, "raw_average_key_size": 28, "raw_value_size": 108044622, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934755, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389846, "oldest_key_time": 0, "file_creation_time": 1732389846, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290254, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:24:09.456375 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290255.sst) /dev/shm/ToplingTemp/Topling-257932-NwQSWN 2024/11/24-03:24:10.459321 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290255.sst): first pass time = 1.00's, 133.829'MB/sec 2024/11/24-03:24:11.850562 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290255.sst): old prealloc_size = 147639500, real_size = 31417184 2024/11/24-03:24:11.851098 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290255.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.01 2024/11/24-03:24:11.851852 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290255.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:24:12.668520 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290255.sst): second pass time = 1.39's, 78.017'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.254'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3416.967'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 279.165'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35340.043'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 78.017'MB/sec zip pipeline throughput = 78.017'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.01 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4443 value = 3.6864 dict = 1.00 all = 3.7937 } Zip/UnZip{ index = 0.0957 value = 0.2713 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2252.816954 GB avg = 0.200 KB (by entry num) total key len = 307.220897 GB avg = 0.027 KB total ukey len = 237.596109 GB avg = 0.015 KB total ukey num = 15.750730250 Billion total entry num = 11.292044198 Billion write speed all = 38.473405304 MB/sec (with seq num) write speed all = 37.115789773 MB/sec (without seq num) 2024/11/24-03:24:12.711401 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290255: 919300 keys, 35378994 bytes, temperature: kWarm 2024/11/24-03:24:12.711466 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389852711433, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290255, "file_size": 35378994, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4276216976, "largest_seqno": 4386048339, "table_properties": {"data_size": 29426768, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389849, "oldest_key_time": 0, "file_creation_time": 1732389849, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290255, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:24:12.745322 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290256.sst) /dev/shm/ToplingTemp/Topling-257932-pDDBUO 2024/11/24-03:24:13.713949 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290256.sst): first pass time = 0.97's, 138.570'MB/sec 2024/11/24-03:24:15.174883 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290256.sst): old prealloc_size = 147639500, real_size = 32237760 2024/11/24-03:24:15.175433 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290256.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931518 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0302 GB avg-val = 116.09 avg-zval = 32.46 2024/11/24-03:24:15.176221 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290256.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:24:16.023632 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290256.sst): second pass time = 1.46's, 74.103'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.828'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3360.512'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 285.627'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35546.537'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.103'MB/sec zip pipeline throughput = 74.103'MB/sec entries = 931518 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 32.46 usrkeys = 931518 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 32.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931518 bytes = 3260320 } seq expand size = 6520626 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5143 value = 3.5768 dict = 1.00 all = 3.6925 } Zip/UnZip{ index = 0.0951 value = 0.2796 dict = 1.00 all = 0.2708 } ---------------------------- total value len = 2252.931610 GB avg = 0.199 KB (by entry num) total key len = 307.246979 GB avg = 0.027 KB total ukey len = 237.614739 GB avg = 0.015 KB total ukey num = 15.751661768 Billion total entry num = 11.292975716 Billion write speed all = 38.473580461 MB/sec (with seq num) write speed all = 37.115921392 MB/sec (without seq num) 2024/11/24-03:24:16.068406 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290256: 931518 keys, 36349218 bytes, temperature: kWarm 2024/11/24-03:24:16.068467 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389856068435, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290256, "file_size": 36349218, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4415795777, "table_properties": {"data_size": 30232816, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26082504, "raw_average_key_size": 28, "raw_value_size": 108135328, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931518, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389852, "oldest_key_time": 0, "file_creation_time": 1732389852, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290256, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:24:16.105855 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290257.sst) /dev/shm/ToplingTemp/Topling-257932-sCAtoN 2024/11/24-03:24:17.031706 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290257.sst): first pass time = 0.93's, 144.972'MB/sec 2024/11/24-03:24:18.289759 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290257.sst): old prealloc_size = 147639500, real_size = 31499792 2024/11/24-03:24:18.290326 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290257.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.10 2024/11/24-03:24:18.291068 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290257.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:24:19.105381 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290257.sst): second pass time = 1.26's, 86.281'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.479'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.986'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 279.149'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34952.967'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 86.281'MB/sec zip pipeline throughput = 86.281'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.10 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4462 value = 3.6760 dict = 1.00 all = 3.7849 } Zip/UnZip{ index = 0.0957 value = 0.2720 dict = 1.00 all = 0.2642 } ---------------------------- total value len = 2253.046522 GB avg = 0.199 KB (by entry num) total key len = 307.272720 GB avg = 0.027 KB total ukey len = 237.633125 GB avg = 0.015 KB total ukey num = 15.752581068 Billion total entry num = 11.293895016 Billion write speed all = 38.473912365 MB/sec (with seq num) write speed all = 37.116205654 MB/sec (without seq num) 2024/11/24-03:24:19.148593 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290257: 919300 keys, 35461618 bytes, temperature: kWarm 2024/11/24-03:24:19.148664 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389859148626, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290257, "file_size": 35461618, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4293484625, "largest_seqno": 4403283415, "table_properties": {"data_size": 29509696, "index_size": 1760064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389856, "oldest_key_time": 0, "file_creation_time": 1732389856, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290257, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:24:19.186927 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290258.sst) /dev/shm/ToplingTemp/Topling-257932-npVGVP 2024/11/24-03:24:20.192411 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290258.sst): first pass time = 1.01's, 133.490'MB/sec 2024/11/24-03:24:21.522689 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290258.sst): old prealloc_size = 147639500, real_size = 32817280 2024/11/24-03:24:21.523270 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290258.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930637 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.22 avg-zval = 33.11 2024/11/24-03:24:21.524055 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290258.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:24:22.382577 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290258.sst): second pass time = 1.33's, 81.385'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.944'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3188.583'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 288.765'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31528.934'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.385'MB/sec zip pipeline throughput = 81.385'MB/sec entries = 930637 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.22 avg-zval = 33.11 usrkeys = 930637 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.22 avg-zval = 33.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930637 bytes = 3257248 } seq expand size = 6514459 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5127 value = 3.5101 dict = 1.00 all = 3.6348 } Zip/UnZip{ index = 0.0951 value = 0.2849 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 2253.161197 GB avg = 0.199 KB (by entry num) total key len = 307.298778 GB avg = 0.027 KB total ukey len = 237.651738 GB avg = 0.015 KB total ukey num = 15.753511705 Billion total entry num = 11.294825653 Billion write speed all = 38.474132436 MB/sec (with seq num) write speed all = 37.116380712 MB/sec (without seq num) 2024/11/24-03:24:22.428502 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290258: 930637 keys, 36925954 bytes, temperature: kWarm 2024/11/24-03:24:22.428574 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389862428538, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290258, "file_size": 36925954, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4415795789, "table_properties": {"data_size": 30813936, "index_size": 1770496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26057836, "raw_average_key_size": 28, "raw_value_size": 108160022, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930637, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389859, "oldest_key_time": 0, "file_creation_time": 1732389859, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290258, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:24:22.465648 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290259.sst) /dev/shm/ToplingTemp/Topling-257932-kEXepM 2024/11/24-03:24:23.328877 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290259.sst): first pass time = 0.86's, 155.490'MB/sec 2024/11/24-03:24:24.580733 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290259.sst): old prealloc_size = 147639500, real_size = 31190256 2024/11/24-03:24:24.581346 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290259.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921238 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.69 avg-zval = 31.69 2024/11/24-03:24:24.582111 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290259.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:24:25.389159 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290259.sst): second pass time = 1.25's, 86.667'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.490'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3010.707'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.339'MB/sec write SST data time = 0.13's, 274.271'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 27653.015'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.667'MB/sec zip pipeline throughput = 86.667'MB/sec entries = 921238 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.69 avg-zval = 31.69 usrkeys = 921238 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.69 avg-zval = 31.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921238 bytes = 3109200 } seq expand size = 6448666 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4454 value = 3.7136 dict = 1.00 all = 3.8177 } Zip/UnZip{ index = 0.0957 value = 0.2693 dict = 1.00 all = 0.2619 } ---------------------------- total value len = 2253.276068 GB avg = 0.199 KB (by entry num) total key len = 307.324572 GB avg = 0.027 KB total ukey len = 237.670163 GB avg = 0.015 KB total ukey num = 15.754432943 Billion total entry num = 11.295746891 Billion write speed all = 38.474507940 MB/sec (with seq num) write speed all = 37.116706817 MB/sec (without seq num) 2024/11/24-03:24:25.432105 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290259: 921238 keys, 35156862 bytes, temperature: kWarm 2024/11/24-03:24:25.432174 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389865432139, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290259, "file_size": 35156862, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4313468109, "largest_seqno": 4415795805, "table_properties": {"data_size": 29195872, "index_size": 1763904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25794664, "raw_average_key_size": 28, "raw_value_size": 108423136, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921238, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389862, "oldest_key_time": 0, "file_creation_time": 1732389862, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290259, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:24:25.468685 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290260.sst) /dev/shm/ToplingTemp/Topling-257932-Eg8zoP 2024/11/24-03:24:26.433710 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290260.sst): first pass time = 0.96's, 139.088'MB/sec 2024/11/24-03:24:27.765938 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290260.sst): old prealloc_size = 147639500, real_size = 32960512 2024/11/24-03:24:27.766528 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290260.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928709 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.52 avg-zval = 33.34 2024/11/24-03:24:27.767329 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290260.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-03:24:28.623580 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290260.sst): second pass time = 1.33's, 81.313'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.218'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3230.103'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 290.271'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34743.490'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.313'MB/sec zip pipeline throughput = 81.313'MB/sec entries = 928709 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.52 avg-zval = 33.34 usrkeys = 928709 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.52 avg-zval = 33.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928709 bytes = 3250496 } seq expand size = 6500963 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5141 value = 3.4951 dict = 1.00 all = 3.6212 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 2253.390783 GB avg = 0.199 KB (by entry num) total key len = 307.350576 GB avg = 0.027 KB total ukey len = 237.688737 GB avg = 0.015 KB total ukey num = 15.755361652 Billion total entry num = 11.296675600 Billion write speed all = 38.474752484 MB/sec (with seq num) write speed all = 37.116905715 MB/sec (without seq num) 2024/11/24-03:24:28.669353 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290260: 928709 keys, 37064096 bytes, temperature: kWarm 2024/11/24-03:24:28.669425 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389868669389, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290260, "file_size": 37064096, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4415796098, "table_properties": {"data_size": 30961584, "index_size": 1766592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003852, "raw_average_key_size": 28, "raw_value_size": 108213990, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928709, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389865, "oldest_key_time": 0, "file_creation_time": 1732389865, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290260, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:24:28.712531 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290261.sst) /dev/shm/ToplingTemp/Topling-257932-5VoipO 2024/11/24-03:24:28.903220 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290261.sst): first pass time = 0.19's, 244.410'MB/sec 2024/11/24-03:24:29.347430 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290261.sst): old prealloc_size = 147639500, real_size = 12573200 2024/11/24-03:24:29.347654 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290261.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 319156 prefix = 4 raw-key = 0.0051 GB zip-key = 0.0006 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0377 GB zip-val = 0.0119 GB avg-val = 118.00 avg-zval = 37.23 2024/11/24-03:24:29.347941 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290261.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0008 GB, waited 0.000 sec, Key+Value = 0.047 GB 2024/11/24-03:24:29.646766 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00/290261.sst): second pass time = 0.44's, 84.920'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.30's, 46.504'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3068.728'MB/sec (index lex order gen) rebuild zvType time = 0.25's, 0.314'MB/sec write SST data time = 0.04's, 309.579'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.386'MB, throughput = 13469.023'MB/sec zip my value time = 0.44's, unzip length = 0.038'GB zip my value throughput = 84.920'MB/sec zip pipeline throughput = 84.920'MB/sec entries = 319156 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 37.23 usrkeys = 319156 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 37.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 319156 bytes = 1037264 } seq expand size = 2234092 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0064 GB value = 0.0377 GB dict = 0.39 MB all = 0.0466 GB } __ZipSize{ index = 0.0006 GB value = 0.0119 GB dict = 0.39 MB all = 0.0139 GB } UnZip/Zip{ index = 10.4327 value = 3.1697 dict = 1.00 all = 3.3477 } Zip/UnZip{ index = 0.0959 value = 0.3155 dict = 1.00 all = 0.2987 } ---------------------------- total value len = 2253.430678 GB avg = 0.199 KB (by entry num) total key len = 307.359513 GB avg = 0.027 KB total ukey len = 237.695120 GB avg = 0.015 KB total ukey num = 15.755680808 Billion total entry num = 11.296994756 Billion write speed all = 38.474894676 MB/sec (with seq num) write speed all = 37.117030420 MB/sec (without seq num) 2024/11/24-03:24:29.669328 139957525694016 [compaction_job.cc:1948] [default] [JOB 40772] Generated table #290261: 319156 keys, 13919227 bytes, temperature: kWarm 2024/11/24-03:24:29.669391 139957525694016 EVENT_LOG_v1 {"time_micros": 1732389869669358, "cf_name": "default", "job": 40772, "event": "table_file_creation", "file_number": 290261, "file_size": 13919227, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4333515582, "largest_seqno": 4371666512, "table_properties": {"data_size": 11881408, "index_size": 611840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 8936368, "raw_average_key_size": 28, "raw_value_size": 37660408, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 319156, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732389868, "oldest_key_time": 0, "file_creation_time": 1732389868, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 290261, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-03:24:29.685080 139957525694016 [compaction_job.cc:703] job-40772: subcompact[0], size: 0.654531 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-40772/att-00 [290243,290244,290245,290246,290247,290248,290249,290250,290251,290252,290253,290254,290255,290256,290257,290258,290259,290260,290261] 2024/11/24-03:24:29.796848 139957525694016 (Original Log Time 2024/11/24-03:23:28.852136) EVENT_LOG_v1 {"time_micros": 1732389808852094, "job": 40772, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [290192], "files_L3": [290225, 290226, 290161, 290162, 290163, 290164, 290165, 290166, 290167, 290168, 290133, 290134, 290135, 290136, 290137, 290138, 290139, 290140, 290141, 290233], "score": 0.695255, "input_data_size": 680909644, "oldest_snapshot_seqno": 4415989299} 2024/11/24-03:24:29.796852 139957525694016 (Original Log Time 2024/11/24-03:24:29.795375) [compaction_job.cc:2022] [default] [JOB 40772] Compacted 1@2 + 20@3 files to L3 => 654531375 bytes 2024/11/24-03:24:29.796854 139957525694016 (Original Log Time 2024/11/24-03:24:29.796774) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 19 0 0 0] max score 0.02, MB/sec: 11.2 rd, 10.8 wr, level 3, files in(1, 20) out(19 +0 blob) MB in(27.1, 622.3 +0.0 blob) out(624.2 +0.0 blob), read-write-amplify(47.0) write-amplify(23.0) OK, records in: 17120541, records dropped: 100896 output_compression: Snappy 2024/11/24-03:24:29.796858 139957525694016 (Original Log Time 2024/11/24-03:24:29.796822) EVENT_LOG_v1 {"time_micros": 1732389869796800, "cf": "default", "job": 40772, "event": "compaction_finished", "compaction_time_micros": 60832952, "compaction_time_cpu_micros": 25509445, "output_level": 3, "num_output_files": 19, "total_output_size": 654531375, "num_input_records": 17120541, "num_output_records": 17019645, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 458703068, "file_range_sync_nanos": 0, "file_fsync_nanos": 13323, "file_prepare_write_nanos": 59157, "lsm_state": [0, 0, 0, 19, 0, 0, 0]} 2024/11/24-03:24:29.918404 139957525694016 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732389808,"executesMs":61079,"compactionJobId":40772,"attempt":0,"compactionInputRawBytes":2476844498,"compactionInputZipBytes":680909644,"compactionOutputRawBytes":0,"compactionOutputZipBytes":654531375,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-9b07c68a984c086edc0141c32fd3fc61-aa7b732d0354d539-00"}}, response = 2024/11/24-03:24:29.918496 139957525694016 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-40772/att-00: olev 3, work 61.079 s, result 7.733 ms, install 1.499 ms, input{raw 2.307 GiB zip 649.366 MiB}