2024/11/22-22:09:03.834764 140025003607616 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00: fork to child time = 0.009280 sec 2024/11/22-22:09:03.835655 140025003607616 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00 2024/11/22-22:09:03.842455 140025003607616 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00 2024/11/22-22:09:03.849774 140025003607616 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-22:09:03.850079 140025003607616 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-22:09:03.897100 140025003607616 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12376/att-00/MANIFEST-206347 2024/11/22-22:09:03.928043 140025003607616 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12376/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 230489, last_sequence is 3598885741, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 230306 2024/11/22-22:09:03.928052 140025003607616 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-22:09:03.928161 140025003607616 [version_set.cc:5721] Creating manifest 230489 2024/11/22-22:09:03.959510 140025003607616 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-22:09:03.983749 140025003607616 [compaction_job.cc:2437] [default] [JOB 12376] Compacting 1@2 + 12@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-22:09:03.983769 140025003607616 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 9310 Base level 2, inputs: [230454(27MB)], [230486(35MB) 230487(5525KB) 230416(36MB) 230417(18MB) 230377(36MB) 230378(34MB) 230379(35MB) 230380(35MB) 230381(34MB) 230382(34MB) 230384(34MB) 230385(28MB)] 2024/11/22-22:09:03.991003 140025003607616 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:09:03.991040 140025003607616 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:09:03.991062 140025003607616 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:09:03.991067 140025003607616 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:09:03.991078 140025003607616 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:09:04.436416 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230490.sst) /dev/shm/ToplingTemp/Topling-309602-pzlzRP 2024/11/22-22:09:05.085956 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230490.sst): first pass time = 0.65's, 206.650'MB/sec 2024/11/22-22:09:06.937823 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230490.sst): old prealloc_size = 147639500, real_size = 32826192 2024/11/22-22:09:06.938404 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230490.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927705 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.68 avg-zval = 33.23 2024/11/22-22:09:06.939254 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230490.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:09:08.313682 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230490.sst): second pass time = 1.85's, 58.479'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.38's, 26.924'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3186.330'MB/sec (index lex order gen) rebuild zvType time = 1.24's, 0.186'MB/sec write SST data time = 0.13's, 281.377'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 39951.023'MB/sec zip my value time = 1.85's, unzip length = 0.108'GB zip my value throughput = 58.479'MB/sec zip pipeline throughput = 58.479'MB/sec entries = 927705 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 33.23 usrkeys = 927705 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 33.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927705 bytes = 3362944 } seq expand size = 6493935 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5134 value = 3.5110 dict = 1.00 all = 3.6233 } Zip/UnZip{ index = 0.0951 value = 0.2848 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 0.114736 GB avg = 0.124 KB (by entry num) total key len = 0.025976 GB avg = 0.028 KB total ukey len = 0.018554 GB avg = 0.020 KB total ukey num = 0.000927705 Billion total entry num = 0.000927705 Billion write speed all = 0.002943438 MB/sec (with seq num) write speed all = 0.002788191 MB/sec (without seq num) 2024/11/22-22:09:08.348846 140025003607616 [compaction_job.cc:1948] [default] [JOB 12376] Generated table #230490: 927705 keys, 37043084 bytes, temperature: kWarm 2024/11/22-22:09:08.348937 140025003607616 EVENT_LOG_v1 {"time_micros": 1732284548348882, "cf_name": "default", "job": 12376, "event": "table_file_creation", "file_number": 230490, "file_size": 37043084, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3598502928, "table_properties": {"data_size": 30829312, "index_size": 1764800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975740, "raw_average_key_size": 28, "raw_value_size": 108242034, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927705, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284544, "oldest_key_time": 0, "file_creation_time": 1732284544, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230490, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:09:08.385309 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230491.sst) /dev/shm/ToplingTemp/Topling-309602-xhyy9R 2024/11/22-22:09:09.479309 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230491.sst): first pass time = 1.09's, 122.690'MB/sec 2024/11/22-22:09:11.394072 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230491.sst): old prealloc_size = 147639500, real_size = 34316608 2024/11/22-22:09:11.394757 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230491.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928967 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.48 avg-zval = 34.79 2024/11/22-22:09:11.395607 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230491.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:09:12.686295 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230491.sst): second pass time = 1.91's, 56.553'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.29's, 29.732'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2625.186'MB/sec (index lex order gen) rebuild zvType time = 1.13's, 0.205'MB/sec write SST data time = 0.16's, 243.051'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34088.871'MB/sec zip my value time = 1.91's, unzip length = 0.108'GB zip my value throughput = 56.553'MB/sec zip pipeline throughput = 56.553'MB/sec entries = 928967 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.48 avg-zval = 34.79 usrkeys = 928967 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.48 avg-zval = 34.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928967 bytes = 3251392 } seq expand size = 6502769 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5148 value = 3.3483 dict = 1.00 all = 3.4933 } Zip/UnZip{ index = 0.0951 value = 0.2987 dict = 1.00 all = 0.2863 } ---------------------------- total value len = 0.229445 GB avg = 0.124 KB (by entry num) total key len = 0.051987 GB avg = 0.028 KB total ukey len = 0.037133 GB avg = 0.020 KB total ukey num = 0.001856672 Billion total entry num = 0.001856672 Billion write speed all = 0.005886523 MB/sec (with seq num) write speed all = 0.005575845 MB/sec (without seq num) 2024/11/22-22:09:12.727240 140025003607616 [compaction_job.cc:1948] [default] [JOB 12376] Generated table #230491: 928967 keys, 38421038 bytes, temperature: kWarm 2024/11/22-22:09:12.727306 140025003607616 EVENT_LOG_v1 {"time_micros": 1732284552727275, "cf_name": "default", "job": 12376, "event": "table_file_creation", "file_number": 230491, "file_size": 38421038, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3598504678, "table_properties": {"data_size": 32317232, "index_size": 1766976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26011076, "raw_average_key_size": 28, "raw_value_size": 108206702, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928967, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284548, "oldest_key_time": 0, "file_creation_time": 1732284548, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230491, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:09:12.765284 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230492.sst) /dev/shm/ToplingTemp/Topling-309602-hllpPQ 2024/11/22-22:09:14.163627 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230492.sst): first pass time = 1.40's, 95.987'MB/sec 2024/11/22-22:09:15.827264 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230492.sst): old prealloc_size = 147639500, real_size = 35059680 2024/11/22-22:09:15.827949 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230492.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929284 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.43 avg-zval = 35.57 2024/11/22-22:09:15.828438 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230492.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:09:16.752827 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230492.sst): second pass time = 1.66's, 65.067'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.521'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2610.264'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.303'MB/sec write SST data time = 0.16's, 237.265'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34546.374'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.067'MB/sec zip pipeline throughput = 65.067'MB/sec entries = 929284 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.43 avg-zval = 35.57 usrkeys = 929284 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.77 avg-zval = 35.57 TagRS{ kinds = RS_Key0_TagN bytes = 145320 } TagArray{ size = 443686 bytes = 1442000 } seq expand size = 3105802 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5123 value = 3.2729 dict = 1.00 all = 3.5788 } Zip/UnZip{ index = 0.0951 value = 0.3055 dict = 1.00 all = 0.2794 } ---------------------------- total value len = 0.340749 GB avg = 0.122 KB (by entry num) total key len = 0.078007 GB avg = 0.028 KB total ukey len = 0.055719 GB avg = 0.020 KB total ukey num = 0.002785956 Billion total entry num = 0.002785956 Billion write speed all = 0.008758081 MB/sec (with seq num) write speed all = 0.008291946 MB/sec (without seq num) 2024/11/22-22:09:16.786356 140025003607616 [compaction_job.cc:1948] [default] [JOB 12376] Generated table #230492: 929284 keys, 37503643 bytes, temperature: kWarm 2024/11/22-22:09:16.786421 140025003607616 EVENT_LOG_v1 {"time_micros": 1732284556786389, "cf_name": "default", "job": 12376, "event": "table_file_creation", "file_number": 230492, "file_size": 37503643, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3598502674, "table_properties": {"data_size": 33059152, "index_size": 1768000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26019952, "raw_average_key_size": 28, "raw_value_size": 108197888, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929284, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284552, "oldest_key_time": 0, "file_creation_time": 1732284552, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230492, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:09:16.822409 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230493.sst) /dev/shm/ToplingTemp/Topling-309602-gGOEbS 2024/11/22-22:09:17.748155 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230493.sst): first pass time = 0.93's, 144.989'MB/sec 2024/11/22-22:09:19.163345 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230493.sst): old prealloc_size = 147639500, real_size = 33623104 2024/11/22-22:09:19.163942 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230493.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920977 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.73 avg-zval = 34.36 2024/11/22-22:09:19.164588 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230493.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:09:20.147494 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230493.sst): second pass time = 1.41's, 76.680'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 37.721'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3049.911'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.270'MB/sec write SST data time = 0.13's, 281.999'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32276.491'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.680'MB/sec zip pipeline throughput = 76.680'MB/sec entries = 920977 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.73 avg-zval = 34.36 usrkeys = 920977 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 34.36 TagRS{ kinds = RS_Key0_TagN bytes = 144000 } TagArray{ size = 712276 bytes = 2492976 } seq expand size = 4985932 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5146 value = 3.4269 dict = 1.00 all = 3.6155 } Zip/UnZip{ index = 0.0951 value = 0.2918 dict = 1.00 all = 0.2766 } ---------------------------- total value len = 0.454165 GB avg = 0.123 KB (by entry num) total key len = 0.103794 GB avg = 0.028 KB total ukey len = 0.074139 GB avg = 0.020 KB total ukey num = 0.003706933 Billion total entry num = 0.003706933 Billion write speed all = 0.011668632 MB/sec (with seq num) write speed all = 0.011048447 MB/sec (without seq num) 2024/11/22-22:09:20.180325 140025003607616 [compaction_job.cc:1948] [default] [JOB 12376] Generated table #230493: 920977 keys, 37122900 bytes, temperature: kWarm 2024/11/22-22:09:20.180390 140025003607616 EVENT_LOG_v1 {"time_micros": 1732284560180358, "cf_name": "default", "job": 12376, "event": "table_file_creation", "file_number": 230493, "file_size": 37122900, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3531006586, "table_properties": {"data_size": 31640880, "index_size": 1751808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25787356, "raw_average_key_size": 28, "raw_value_size": 108430414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920977, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284556, "oldest_key_time": 0, "file_creation_time": 1732284556, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230493, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:09:20.216538 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230494.sst) /dev/shm/ToplingTemp/Topling-309602-iB10hR 2024/11/22-22:09:21.020940 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230494.sst): first pass time = 0.80's, 166.862'MB/sec 2024/11/22-22:09:22.414582 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230494.sst): old prealloc_size = 147639500, real_size = 33491552 2024/11/22-22:09:22.415183 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230494.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924859 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.12 avg-zval = 34.06 2024/11/22-22:09:22.415942 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230494.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:09:23.447839 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230494.sst): second pass time = 1.39's, 77.767'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 36.264'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3052.616'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.257'MB/sec write SST data time = 0.13's, 284.427'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31622.671'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.767'MB/sec zip pipeline throughput = 77.767'MB/sec entries = 924859 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.12 avg-zval = 34.06 usrkeys = 924859 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.12 avg-zval = 34.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924859 bytes = 3121408 } seq expand size = 6474013 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5136 value = 3.4387 dict = 1.00 all = 3.5821 } Zip/UnZip{ index = 0.0951 value = 0.2908 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 0.568961 GB avg = 0.123 KB (by entry num) total key len = 0.129690 GB avg = 0.028 KB total ukey len = 0.092636 GB avg = 0.020 KB total ukey num = 0.004631792 Billion total entry num = 0.004631792 Billion write speed all = 0.014609918 MB/sec (with seq num) write speed all = 0.013835052 MB/sec (without seq num) 2024/11/22-22:09:23.481611 140025003607616 [compaction_job.cc:1948] [default] [JOB 12376] Generated table #230494: 924859 keys, 37468874 bytes, temperature: kWarm 2024/11/22-22:09:23.481681 140025003607616 EVENT_LOG_v1 {"time_micros": 1732284563481644, "cf_name": "default", "job": 12376, "event": "table_file_creation", "file_number": 230494, "file_size": 37468874, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3598503148, "table_properties": {"data_size": 31500816, "index_size": 1759360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25896052, "raw_average_key_size": 28, "raw_value_size": 108321730, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924859, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284560, "oldest_key_time": 0, "file_creation_time": 1732284560, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230494, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:09:23.517691 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230495.sst) /dev/shm/ToplingTemp/Topling-309602-VS9hhS 2024/11/22-22:09:24.318985 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230495.sst): first pass time = 0.80's, 167.510'MB/sec 2024/11/22-22:09:25.693902 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230495.sst): old prealloc_size = 147639500, real_size = 32870928 2024/11/22-22:09:25.694486 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230495.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930225 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.29 avg-zval = 33.17 2024/11/22-22:09:25.695274 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230495.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:09:26.738608 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230495.sst): second pass time = 1.37's, 78.714'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 35.397'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3189.595'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.13's, 282.203'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30801.051'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.714'MB/sec zip pipeline throughput = 78.714'MB/sec entries = 930225 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.29 avg-zval = 33.17 usrkeys = 930225 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.29 avg-zval = 33.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930225 bytes = 3255808 } seq expand size = 6511575 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4443 value = 3.5056 dict = 1.00 all = 3.6296 } Zip/UnZip{ index = 0.0957 value = 0.2853 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 0.683644 GB avg = 0.123 KB (by entry num) total key len = 0.155736 GB avg = 0.028 KB total ukey len = 0.111240 GB avg = 0.020 KB total ukey num = 0.005562017 Billion total entry num = 0.005562017 Billion write speed all = 0.017551585 MB/sec (with seq num) write speed all = 0.016621164 MB/sec (without seq num) 2024/11/22-22:09:26.771135 140025003607616 [compaction_job.cc:1948] [default] [JOB 12376] Generated table #230495: 930225 keys, 36978914 bytes, temperature: kWarm 2024/11/22-22:09:26.771201 140025003607616 EVENT_LOG_v1 {"time_micros": 1732284566771168, "cf_name": "default", "job": 12376, "event": "table_file_creation", "file_number": 230495, "file_size": 36978914, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3438862052, "largest_seqno": 3598504654, "table_properties": {"data_size": 30856896, "index_size": 1781312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26046300, "raw_average_key_size": 28, "raw_value_size": 108171458, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930225, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284563, "oldest_key_time": 0, "file_creation_time": 1732284563, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230495, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:09:26.823238 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230496.sst) /dev/shm/ToplingTemp/Topling-309602-wywK8R 2024/11/22-22:09:27.739988 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230496.sst): first pass time = 0.92's, 146.413'MB/sec 2024/11/22-22:09:29.081984 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230496.sst): old prealloc_size = 147639500, real_size = 32993184 2024/11/22-22:09:29.082542 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230496.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924764 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.14 avg-zval = 33.53 2024/11/22-22:09:29.083328 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230496.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:09:30.084896 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230496.sst): second pass time = 1.34's, 80.766'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 36.978'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3277.944'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.265'MB/sec write SST data time = 0.13's, 283.492'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32767.580'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.766'MB/sec zip pipeline throughput = 80.766'MB/sec entries = 924764 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.14 avg-zval = 33.53 usrkeys = 924764 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.14 avg-zval = 33.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924764 bytes = 3236688 } seq expand size = 6473348 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5144 value = 3.4940 dict = 1.00 all = 3.6191 } Zip/UnZip{ index = 0.0951 value = 0.2862 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 0.798442 GB avg = 0.123 KB (by entry num) total key len = 0.181630 GB avg = 0.028 KB total ukey len = 0.129736 GB avg = 0.020 KB total ukey num = 0.006486781 Billion total entry num = 0.006486781 Billion write speed all = 0.020492025 MB/sec (with seq num) write speed all = 0.019406984 MB/sec (without seq num) 2024/11/22-22:09:30.116838 140025003607616 [compaction_job.cc:1948] [default] [JOB 12376] Generated table #230496: 924764 keys, 37085564 bytes, temperature: kWarm 2024/11/22-22:09:30.116906 140025003607616 EVENT_LOG_v1 {"time_micros": 1732284570116872, "cf_name": "default", "job": 12376, "event": "table_file_creation", "file_number": 230496, "file_size": 37085564, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3440179021, "largest_seqno": 3598503331, "table_properties": {"data_size": 31002768, "index_size": 1759040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25893392, "raw_average_key_size": 28, "raw_value_size": 108324384, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924764, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284566, "oldest_key_time": 0, "file_creation_time": 1732284566, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230496, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:09:30.158971 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230497.sst) /dev/shm/ToplingTemp/Topling-309602-QOYfMS 2024/11/22-22:09:31.065952 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230497.sst): first pass time = 0.91's, 147.990'MB/sec 2024/11/22-22:09:32.431349 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230497.sst): old prealloc_size = 147639500, real_size = 33543712 2024/11/22-22:09:32.431922 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230497.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924662 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.15 avg-zval = 34.12 2024/11/22-22:09:32.432685 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230497.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:09:33.472851 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230497.sst): second pass time = 1.36's, 79.376'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 36.137'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3204.016'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.254'MB/sec write SST data time = 0.13's, 287.578'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31425.998'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.376'MB/sec zip pipeline throughput = 79.376'MB/sec entries = 924662 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 34.12 usrkeys = 924662 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 34.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924662 bytes = 3236336 } seq expand size = 6472634 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5144 value = 3.4331 dict = 1.00 all = 3.5662 } Zip/UnZip{ index = 0.0951 value = 0.2913 dict = 1.00 all = 0.2804 } ---------------------------- total value len = 0.913242 GB avg = 0.123 KB (by entry num) total key len = 0.207520 GB avg = 0.028 KB total ukey len = 0.148229 GB avg = 0.020 KB total ukey num = 0.007411443 Billion total entry num = 0.007411443 Billion write speed all = 0.023432018 MB/sec (with seq num) write speed all = 0.022192397 MB/sec (without seq num) 2024/11/22-22:09:33.508482 140025003607616 [compaction_job.cc:1948] [default] [JOB 12376] Generated table #230497: 924662 keys, 37636040 bytes, temperature: kWarm 2024/11/22-22:09:33.508556 140025003607616 EVENT_LOG_v1 {"time_micros": 1732284573508521, "cf_name": "default", "job": 12376, "event": "table_file_creation", "file_number": 230497, "file_size": 37636040, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268533, "largest_seqno": 3598503432, "table_properties": {"data_size": 31553488, "index_size": 1758848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25890536, "raw_average_key_size": 28, "raw_value_size": 108327220, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924662, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284570, "oldest_key_time": 0, "file_creation_time": 1732284570, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230497, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:09:33.544561 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230498.sst) /dev/shm/ToplingTemp/Topling-309602-5DNZ3O 2024/11/22-22:09:34.430755 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230498.sst): first pass time = 0.89's, 151.461'MB/sec 2024/11/22-22:09:35.830265 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230498.sst): old prealloc_size = 147639500, real_size = 34603392 2024/11/22-22:09:35.830866 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230498.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927346 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 116.73 avg-zval = 35.16 2024/11/22-22:09:35.831618 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230498.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:09:36.843149 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230498.sst): second pass time = 1.40's, 77.387'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 38.212'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3049.331'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.264'MB/sec write SST data time = 0.13's, 291.057'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32584.117'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.387'MB/sec zip pipeline throughput = 77.387'MB/sec entries = 927346 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.73 avg-zval = 35.16 usrkeys = 927346 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.73 avg-zval = 35.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927346 bytes = 3245728 } seq expand size = 6491422 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5120 value = 3.3199 dict = 1.00 all = 3.4678 } Zip/UnZip{ index = 0.0951 value = 0.3012 dict = 1.00 all = 0.2884 } ---------------------------- total value len = 1.027985 GB avg = 0.123 KB (by entry num) total key len = 0.233486 GB avg = 0.028 KB total ukey len = 0.166776 GB avg = 0.020 KB total ukey num = 0.008338789 Billion total entry num = 0.008338789 Billion write speed all = 0.026371997 MB/sec (with seq num) write speed all = 0.024977368 MB/sec (without seq num) 2024/11/22-22:09:36.877070 140025003607616 [compaction_job.cc:1948] [default] [JOB 12376] Generated table #230498: 927346 keys, 38703870 bytes, temperature: kWarm 2024/11/22-22:09:36.877140 140025003607616 EVENT_LOG_v1 {"time_micros": 1732284576877104, "cf_name": "default", "job": 12376, "event": "table_file_creation", "file_number": 230498, "file_size": 38703870, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3598504773, "table_properties": {"data_size": 32607024, "index_size": 1764352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25965688, "raw_average_key_size": 28, "raw_value_size": 108252060, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927346, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284573, "oldest_key_time": 0, "file_creation_time": 1732284573, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230498, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:09:36.918282 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230499.sst) /dev/shm/ToplingTemp/Topling-309602-T5impQ 2024/11/22-22:09:37.732794 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230499.sst): first pass time = 0.81's, 164.792'MB/sec 2024/11/22-22:09:39.074090 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230499.sst): old prealloc_size = 147639500, real_size = 31703056 2024/11/22-22:09:39.074627 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230499.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924636 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 117.16 avg-zval = 32.13 2024/11/22-22:09:39.075406 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230499.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:09:40.049614 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230499.sst): second pass time = 1.34's, 80.836'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 36.694'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3402.001'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.274'MB/sec write SST data time = 0.13's, 272.866'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 24777.126'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.836'MB/sec zip pipeline throughput = 80.836'MB/sec entries = 924636 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.16 avg-zval = 32.13 usrkeys = 924636 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.16 avg-zval = 32.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924636 bytes = 3236240 } seq expand size = 6472452 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5122 value = 3.6459 dict = 1.00 all = 3.7496 } Zip/UnZip{ index = 0.0951 value = 0.2743 dict = 1.00 all = 0.2667 } ---------------------------- total value len = 1.142786 GB avg = 0.123 KB (by entry num) total key len = 0.259376 GB avg = 0.028 KB total ukey len = 0.185269 GB avg = 0.020 KB total ukey num = 0.009263425 Billion total entry num = 0.009263425 Billion write speed all = 0.029311266 MB/sec (with seq num) write speed all = 0.027762100 MB/sec (without seq num) 2024/11/22-22:09:40.081441 140025003607616 [compaction_job.cc:1948] [default] [JOB 12376] Generated table #230499: 924636 keys, 35795472 bytes, temperature: kWarm 2024/11/22-22:09:40.081508 140025003607616 EVENT_LOG_v1 {"time_micros": 1732284580081475, "cf_name": "default", "job": 12376, "event": "table_file_creation", "file_number": 230499, "file_size": 35795472, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3430731566, "largest_seqno": 3598503259, "table_properties": {"data_size": 29712576, "index_size": 1759168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25889808, "raw_average_key_size": 28, "raw_value_size": 108328012, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924636, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284576, "oldest_key_time": 0, "file_creation_time": 1732284576, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230499, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:09:40.116762 140025003607616 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230500.sst) /dev/shm/ToplingTemp/Topling-309602-b58oaP 2024/11/22-22:09:40.441226 140025003607616 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230500.sst): first pass time = 0.32's, 241.753'MB/sec 2024/11/22-22:09:41.212589 140025003607616 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230500.sst): old prealloc_size = 147639500, real_size = 19370368 2024/11/22-22:09:41.212976 140025003607616 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230500.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 542273 prefix = 4 raw-key = 0.0087 GB zip-key = 0.0010 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0632 GB zip-val = 0.0182 GB avg-val = 116.56 avg-zval = 33.55 2024/11/22-22:09:41.213430 140025003607616 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230500.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0014 GB, waited 0.000 sec, Key+Value = 0.078 GB 2024/11/22-22:09:41.808663 140025003607616 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00/230500.sst): second pass time = 0.77's, 82.054'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.60's, 36.297'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2856.178'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.261'MB/sec write SST data time = 0.08's, 281.799'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.635'MB, throughput = 18466.097'MB/sec zip my value time = 0.77's, unzip length = 0.063'GB zip my value throughput = 82.054'MB/sec zip pipeline throughput = 82.054'MB/sec entries = 542273 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.56 avg-zval = 33.55 usrkeys = 542273 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.56 avg-zval = 33.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 542273 bytes = 1762400 } seq expand size = 3795911 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0108 GB value = 0.0632 GB dict = 0.64 MB all = 0.0784 GB } __ZipSize{ index = 0.0010 GB value = 0.0182 GB dict = 0.64 MB all = 0.0216 GB } UnZip/Zip{ index = 10.4392 value = 3.4738 dict = 1.00 all = 3.6234 } Zip/UnZip{ index = 0.0958 value = 0.2879 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 1.209789 GB avg = 0.123 KB (by entry num) total key len = 0.274560 GB avg = 0.028 KB total ukey len = 0.196114 GB avg = 0.020 KB total ukey num = 0.009805698 Billion total entry num = 0.009805698 Billion write speed all = 0.031028197 MB/sec (with seq num) write speed all = 0.029388404 MB/sec (without seq num) 2024/11/22-22:09:41.830109 140025003607616 [compaction_job.cc:1948] [default] [JOB 12376] Generated table #230500: 542273 keys, 21634525 bytes, temperature: kWarm 2024/11/22-22:09:41.830173 140025003607616 EVENT_LOG_v1 {"time_micros": 1732284581830141, "cf_name": "default", "job": 12376, "event": "table_file_creation", "file_number": 230500, "file_size": 21634525, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3532145335, "largest_seqno": 3598503277, "table_properties": {"data_size": 18195696, "index_size": 1038912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 15183644, "raw_average_key_size": 28, "raw_value_size": 63207742, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 542273, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732284580, "oldest_key_time": 0, "file_creation_time": 1732284580, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 230500, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:09:41.849445 140025003607616 [compaction_job.cc:703] job-12376: subcompact[0], size: 0.395394 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-12376/att-00 [230490,230491,230492,230493,230494,230495,230496,230497,230498,230499,230500] 2024/11/22-22:09:41.924547 140025003607616 (Original Log Time 2024/11/22-22:09:03.983924) EVENT_LOG_v1 {"time_micros": 1732284543983788, "job": 12376, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [230454], "files_L3": [230486, 230487, 230416, 230417, 230377, 230378, 230379, 230380, 230381, 230382, 230384, 230385], "score": 0.999251, "input_data_size": 415953991, "oldest_snapshot_seqno": 3598885659} 2024/11/22-22:09:41.924550 140025003607616 (Original Log Time 2024/11/22-22:09:41.923902) [compaction_job.cc:2022] [default] [JOB 12376] Compacted 1@2 + 12@3 files to L3 => 395393924 bytes 2024/11/22-22:09:41.924552 140025003607616 (Original Log Time 2024/11/22-22:09:41.924479) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 11.0 rd, 10.4 wr, level 3, files in(1, 12) out(11 +0 blob) MB in(27.1, 369.6 +0.0 blob) out(377.1 +0.0 blob), read-write-amplify(28.5) write-amplify(13.9) OK, records in: 9894897, records dropped: 89199 output_compression: Snappy 2024/11/22-22:09:41.924558 140025003607616 (Original Log Time 2024/11/22-22:09:41.924526) EVENT_LOG_v1 {"time_micros": 1732284581924503, "cf": "default", "job": 12376, "event": "compaction_finished", "compaction_time_micros": 37865532, "compaction_time_cpu_micros": 17514069, "output_level": 3, "num_output_files": 11, "total_output_size": 395393924, "num_input_records": 9894897, "num_output_records": 9805698, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 254548374, "file_range_sync_nanos": 0, "file_fsync_nanos": 9497, "file_prepare_write_nanos": 23869, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/22-22:09:42.044793 140025003607616 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732284543,"executesMs":38082,"compactionJobId":12376,"attempt":0,"compactionInputRawBytes":1433235446,"compactionInputZipBytes":415953991,"compactionOutputRawBytes":0,"compactionOutputZipBytes":395393924,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-fc716a8e00871e2982c4187ed7fbe124-4970733813fbe056-00"}}, response = 2024/11/22-22:09:42.044875 140025003607616 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-12376/att-00: olev 3, work 38.083 s, result 6.399 ms, install 0.675 ms, input{raw 1.335 GiB zip 396.685 MiB}