2024/11/23-04:05:37.103248 140025121105472 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00: fork to child time = 0.020251 sec 2024/11/23-04:05:37.104103 140025121105472 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00 2024/11/23-04:05:37.110510 140025121105472 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00 2024/11/23-04:05:37.111635 140025121105472 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-04:05:37.111887 140025121105472 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-04:05:37.161442 140025121105472 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-18936/att-00/MANIFEST-206347 2024/11/23-04:05:37.174740 140025121105472 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-18936/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 243943, last_sequence is 3775993718, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 243757 2024/11/23-04:05:37.174745 140025121105472 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-04:05:37.174853 140025121105472 [version_set.cc:5721] Creating manifest 243943 2024/11/23-04:05:37.205584 140025121105472 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-04:05:37.239261 140025121105472 [compaction_job.cc:2437] [default] [JOB 18936] Compacting 1@2 + 12@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-04:05:37.239278 140025121105472 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 14153 Base level 2, inputs: [243894(27MB)], [243940(35MB) 243941(5220KB) 243867(35MB) 243868(34MB) 243869(21MB) 243821(35MB) 243822(34MB) 243823(37MB) 243824(35MB) 243825(35MB) 243826(35MB) 243827(19MB)] 2024/11/23-04:05:37.245288 140025121105472 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-04:05:37.245328 140025121105472 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-04:05:37.245356 140025121105472 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-04:05:37.245364 140025121105472 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-04:05:37.245376 140025121105472 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-04:05:37.680970 140025121105472 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243944.sst) /dev/shm/ToplingTemp/Topling-403519-TMf5Gn 2024/11/23-04:05:38.755608 140025121105472 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243944.sst): first pass time = 1.07's, 124.901'MB/sec 2024/11/23-04:05:40.676717 140025121105472 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243944.sst): old prealloc_size = 147639500, real_size = 32619088 2024/11/23-04:05:40.677267 140025121105472 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243944.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933254 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.82 avg-zval = 32.79 2024/11/23-04:05:40.678094 140025121105472 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243944.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:05:41.947429 140025121105472 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243944.sst): second pass time = 1.92's, 56.315'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.27's, 29.001'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3381.281'MB/sec (index lex order gen) rebuild zvType time = 1.14's, 0.204'MB/sec write SST data time = 0.13's, 285.918'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35586.580'MB/sec zip my value time = 1.92's, unzip length = 0.108'GB zip my value throughput = 56.315'MB/sec zip pipeline throughput = 62.578'MB/sec entries = 933254 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.82 avg-zval = 32.79 usrkeys = 933254 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.82 avg-zval = 32.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933254 bytes = 3383056 } seq expand size = 6532778 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4438 value = 3.5324 dict = 1.00 all = 3.6420 } Zip/UnZip{ index = 0.0958 value = 0.2831 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 0.114619 GB avg = 0.123 KB (by entry num) total key len = 0.026131 GB avg = 0.028 KB total ukey len = 0.018665 GB avg = 0.020 KB total ukey num = 0.000933254 Billion total entry num = 0.000933254 Billion write speed all = 0.002034002 MB/sec (with seq num) write speed all = 0.001926109 MB/sec (without seq num) 2024/11/23-04:05:41.982832 140025121105472 [compaction_job.cc:1948] [default] [JOB 18936] Generated table #243944: 933254 keys, 36852440 bytes, temperature: kWarm 2024/11/23-04:05:41.982917 140025121105472 EVENT_LOG_v1 {"time_micros": 1732305941982864, "cf_name": "default", "job": 18936, "event": "table_file_creation", "file_number": 243944, "file_size": 36852440, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3339428666, "largest_seqno": 3775663239, "table_properties": {"data_size": 30598400, "index_size": 1787200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26131112, "raw_average_key_size": 28, "raw_value_size": 108086692, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933254, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305937, "oldest_key_time": 0, "file_creation_time": 1732305937, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243944, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:05:42.020375 140025121105472 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243945.sst) /dev/shm/ToplingTemp/Topling-403519-ltDTTn 2024/11/23-04:05:43.012887 140025121105472 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243945.sst): first pass time = 0.99's, 135.235'MB/sec 2024/11/23-04:05:45.127323 140025121105472 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243945.sst): old prealloc_size = 147639500, real_size = 33109792 2024/11/23-04:05:45.127942 140025121105472 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243945.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927636 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.69 avg-zval = 33.54 2024/11/23-04:05:45.128801 140025121105472 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243945.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:05:46.421571 140025121105472 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243945.sst): second pass time = 2.10's, 51.483'MB/sec, value only(80.6% of KV) wait indexing time = 0.01's, remap KeyValue time = 1.29's, 28.751'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2958.078'MB/sec (index lex order gen) rebuild zvType time = 1.07's, 0.216'MB/sec write SST data time = 0.22's, 168.771'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 23209.033'MB/sec zip my value time = 2.10's, unzip length = 0.108'GB zip my value throughput = 51.483'MB/sec zip pipeline throughput = 84.484'MB/sec entries = 927636 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.69 avg-zval = 33.54 usrkeys = 927636 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.69 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927636 bytes = 3246736 } seq expand size = 6493452 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5130 value = 3.4791 dict = 1.00 all = 3.6070 } Zip/UnZip{ index = 0.0951 value = 0.2874 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 0.229357 GB avg = 0.123 KB (by entry num) total key len = 0.052105 GB avg = 0.028 KB total ukey len = 0.037218 GB avg = 0.020 KB total ukey num = 0.001860890 Billion total entry num = 0.001860890 Billion write speed all = 0.004067173 MB/sec (with seq num) write speed all = 0.003852051 MB/sec (without seq num) 2024/11/23-04:05:46.476003 140025121105472 [compaction_job.cc:1948] [default] [JOB 18936] Generated table #243945: 927636 keys, 37210666 bytes, temperature: kWarm 2024/11/23-04:05:46.476053 140025121105472 EVENT_LOG_v1 {"time_micros": 1732305946476026, "cf_name": "default", "job": 18936, "event": "table_file_creation", "file_number": 243945, "file_size": 37210666, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 3775663056, "table_properties": {"data_size": 31112976, "index_size": 1764736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25973808, "raw_average_key_size": 28, "raw_value_size": 108244036, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927636, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305942, "oldest_key_time": 0, "file_creation_time": 1732305942, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243945, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:05:46.518345 140025121105472 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243946.sst) /dev/shm/ToplingTemp/Topling-403519-uTuber 2024/11/23-04:05:47.508425 140025121105472 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243946.sst): first pass time = 0.99's, 135.567'MB/sec 2024/11/23-04:05:49.081025 140025121105472 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243946.sst): old prealloc_size = 147639500, real_size = 32756368 2024/11/23-04:05:49.081618 140025121105472 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243946.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930988 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.17 avg-zval = 33.03 2024/11/23-04:05:49.082403 140025121105472 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243946.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:05:50.049831 140025121105472 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243946.sst): second pass time = 1.57's, 68.819'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.97's, 38.175'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3109.049'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.279'MB/sec write SST data time = 0.13's, 279.095'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30746.022'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.819'MB/sec zip pipeline throughput = 125.412'MB/sec entries = 930988 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.17 avg-zval = 33.03 usrkeys = 930988 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.17 avg-zval = 33.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930988 bytes = 3374848 } seq expand size = 6516916 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5118 value = 3.5168 dict = 1.00 all = 3.6292 } Zip/UnZip{ index = 0.0951 value = 0.2843 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 0.344024 GB avg = 0.123 KB (by entry num) total key len = 0.078173 GB avg = 0.028 KB total ukey len = 0.055838 GB avg = 0.020 KB total ukey num = 0.002791878 Billion total entry num = 0.002791878 Billion write speed all = 0.006100493 MB/sec (with seq num) write speed all = 0.005777765 MB/sec (without seq num) 2024/11/23-04:05:50.091407 140025121105472 [compaction_job.cc:1948] [default] [JOB 18936] Generated table #243946: 930988 keys, 36982476 bytes, temperature: kWarm 2024/11/23-04:05:50.091456 140025121105472 EVENT_LOG_v1 {"time_micros": 1732305950091430, "cf_name": "default", "job": 18936, "event": "table_file_creation", "file_number": 243946, "file_size": 36982476, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3775663936, "table_properties": {"data_size": 30752128, "index_size": 1771328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26067664, "raw_average_key_size": 28, "raw_value_size": 108150108, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930988, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305946, "oldest_key_time": 0, "file_creation_time": 1732305946, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243946, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:05:50.135131 140025121105472 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243947.sst) /dev/shm/ToplingTemp/Topling-403519-ToOBfp 2024/11/23-04:05:51.174593 140025121105472 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243947.sst): first pass time = 1.04's, 129.126'MB/sec 2024/11/23-04:05:52.614375 140025121105472 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243947.sst): old prealloc_size = 147639500, real_size = 34044176 2024/11/23-04:05:52.614947 140025121105472 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243947.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930876 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0320 GB avg-val = 116.18 avg-zval = 34.39 2024/11/23-04:05:52.615656 140025121105472 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243947.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:05:53.505418 140025121105472 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243947.sst): second pass time = 1.44's, 75.198'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 42.374'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.639'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.13's, 289.384'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35506.966'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.198'MB/sec zip pipeline throughput = 75.198'MB/sec entries = 930876 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.18 avg-zval = 34.39 usrkeys = 930876 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.81 avg-zval = 34.39 TagRS{ kinds = RS_Key0_TagN bytes = 145560 } TagArray{ size = 747879 bytes = 2711072 } seq expand size = 5235153 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.3789 value = 3.3779 dict = 1.00 all = 3.5549 } Zip/UnZip{ index = 0.0963 value = 0.2960 dict = 1.00 all = 0.2813 } ---------------------------- total value len = 0.457412 GB avg = 0.123 KB (by entry num) total key len = 0.104237 GB avg = 0.028 KB total ukey len = 0.074455 GB avg = 0.020 KB total ukey num = 0.003722754 Billion total entry num = 0.003722754 Billion write speed all = 0.008115101 MB/sec (with seq num) write speed all = 0.007684790 MB/sec (without seq num) 2024/11/23-04:05:53.538576 140025121105472 [compaction_job.cc:1948] [default] [JOB 18936] Generated table #243947: 930876 keys, 37756066 bytes, temperature: kWarm 2024/11/23-04:05:53.538641 140025121105472 EVENT_LOG_v1 {"time_micros": 1732305953538607, "cf_name": "default", "job": 18936, "event": "table_file_creation", "file_number": 243947, "file_size": 37756066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3775663761, "table_properties": {"data_size": 32017472, "index_size": 1793792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26064528, "raw_average_key_size": 28, "raw_value_size": 108153216, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930876, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305950, "oldest_key_time": 0, "file_creation_time": 1732305950, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243947, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:05:53.573983 140025121105472 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243948.sst) /dev/shm/ToplingTemp/Topling-403519-3l2pFq 2024/11/23-04:05:54.370896 140025121105472 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243948.sst): first pass time = 0.80's, 168.430'MB/sec 2024/11/23-04:05:55.763061 140025121105472 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243948.sst): old prealloc_size = 147639500, real_size = 31785408 2024/11/23-04:05:55.763609 140025121105472 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243948.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928601 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0298 GB avg-val = 116.54 avg-zval = 32.08 2024/11/23-04:05:55.764423 140025121105472 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243948.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:05:56.663450 140025121105472 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243948.sst): second pass time = 1.39's, 77.770'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 39.989'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3363.745'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.13's, 280.433'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29000.242'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.770'MB/sec zip pipeline throughput = 77.770'MB/sec entries = 928601 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.54 avg-zval = 32.08 usrkeys = 928601 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.54 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928601 bytes = 3366192 } seq expand size = 6500207 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5129 value = 3.6331 dict = 1.00 all = 3.7278 } Zip/UnZip{ index = 0.0951 value = 0.2752 dict = 1.00 all = 0.2683 } ---------------------------- total value len = 0.572130 GB avg = 0.123 KB (by entry num) total key len = 0.130238 GB avg = 0.028 KB total ukey len = 0.093027 GB avg = 0.020 KB total ukey num = 0.004651355 Billion total entry num = 0.004651355 Billion write speed all = 0.010147829 MB/sec (with seq num) write speed all = 0.009610206 MB/sec (without seq num) 2024/11/23-04:05:56.696121 140025121105472 [compaction_job.cc:1948] [default] [JOB 18936] Generated table #243948: 928601 keys, 36004856 bytes, temperature: kWarm 2024/11/23-04:05:56.696184 140025121105472 EVENT_LOG_v1 {"time_micros": 1732305956696151, "cf_name": "default", "job": 18936, "event": "table_file_creation", "file_number": 243948, "file_size": 36004856, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3775662904, "table_properties": {"data_size": 29786480, "index_size": 1766592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26000828, "raw_average_key_size": 28, "raw_value_size": 108216978, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928601, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305953, "oldest_key_time": 0, "file_creation_time": 1732305953, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243948, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:05:56.732701 140025121105472 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243949.sst) /dev/shm/ToplingTemp/Topling-403519-BT7ELn 2024/11/23-04:05:57.639019 140025121105472 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243949.sst): first pass time = 0.91's, 148.098'MB/sec 2024/11/23-04:05:59.127841 140025121105472 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243949.sst): old prealloc_size = 147639500, real_size = 32648192 2024/11/23-04:05:59.128359 140025121105472 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243949.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929840 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.35 avg-zval = 32.93 2024/11/23-04:05:59.129152 140025121105472 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243949.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:06:00.029599 140025121105472 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243949.sst): second pass time = 1.49's, 72.697'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.760'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3626.782'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.300'MB/sec write SST data time = 0.13's, 289.267'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31777.804'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.697'MB/sec zip pipeline throughput = 103.276'MB/sec entries = 929840 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.35 avg-zval = 32.93 usrkeys = 929840 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.35 avg-zval = 32.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929840 bytes = 3254448 } seq expand size = 6508880 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.3788 value = 3.5326 dict = 1.00 all = 3.6517 } Zip/UnZip{ index = 0.0964 value = 0.2831 dict = 1.00 all = 0.2738 } ---------------------------- total value len = 0.686821 GB avg = 0.123 KB (by entry num) total key len = 0.156273 GB avg = 0.028 KB total ukey len = 0.111624 GB avg = 0.020 KB total ukey num = 0.005581195 Billion total entry num = 0.005581195 Billion write speed all = 0.012180461 MB/sec (with seq num) write speed all = 0.011535394 MB/sec (without seq num) 2024/11/23-04:06:00.059951 140025121105472 [compaction_job.cc:1948] [default] [JOB 18936] Generated table #243949: 929840 keys, 36755126 bytes, temperature: kWarm 2024/11/23-04:06:00.060017 140025121105472 EVENT_LOG_v1 {"time_micros": 1732305960059982, "cf_name": "default", "job": 18936, "event": "table_file_creation", "file_number": 243949, "file_size": 36755126, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 3775662826, "table_properties": {"data_size": 30623728, "index_size": 1791808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26035520, "raw_average_key_size": 28, "raw_value_size": 108182260, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929840, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305956, "oldest_key_time": 0, "file_creation_time": 1732305956, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243949, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:06:00.096078 140025121105472 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243950.sst) /dev/shm/ToplingTemp/Topling-403519-dXYzUr 2024/11/23-04:06:00.906730 140025121105472 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243950.sst): first pass time = 0.81's, 165.575'MB/sec 2024/11/23-04:06:02.454003 140025121105472 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243950.sst): old prealloc_size = 147639500, real_size = 34370992 2024/11/23-04:06:02.454546 140025121105472 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243950.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926521 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 116.86 avg-zval = 34.94 2024/11/23-04:06:02.455321 140025121105472 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243950.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:06:03.375619 140025121105472 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243950.sst): second pass time = 1.55's, 70.029'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.866'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3391.627'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.13's, 300.343'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25743.785'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 70.029'MB/sec zip pipeline throughput = 99.291'MB/sec entries = 926521 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.86 avg-zval = 34.94 usrkeys = 926521 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.86 avg-zval = 34.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926521 bytes = 3358656 } seq expand size = 6485647 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5111 value = 3.3443 dict = 1.00 all = 3.4786 } Zip/UnZip{ index = 0.0951 value = 0.2990 dict = 1.00 all = 0.2875 } ---------------------------- total value len = 0.801582 GB avg = 0.123 KB (by entry num) total key len = 0.182216 GB avg = 0.028 KB total ukey len = 0.130154 GB avg = 0.020 KB total ukey num = 0.006507716 Billion total entry num = 0.006507716 Billion write speed all = 0.014212563 MB/sec (with seq num) write speed all = 0.013460446 MB/sec (without seq num) 2024/11/23-04:06:03.411337 140025121105472 [compaction_job.cc:1948] [default] [JOB 18936] Generated table #243950: 926521 keys, 38584020 bytes, temperature: kWarm 2024/11/23-04:06:03.411403 140025121105472 EVENT_LOG_v1 {"time_micros": 1732305963411370, "cf_name": "default", "job": 18936, "event": "table_file_creation", "file_number": 243950, "file_size": 38584020, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 3775662025, "table_properties": {"data_size": 32376224, "index_size": 1762944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25942588, "raw_average_key_size": 28, "raw_value_size": 108275182, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926521, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305960, "oldest_key_time": 0, "file_creation_time": 1732305960, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243950, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:06:03.449866 140025121105472 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243951.sst) /dev/shm/ToplingTemp/Topling-403519-qadCro 2024/11/23-04:06:04.252743 140025121105472 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243951.sst): first pass time = 0.80's, 167.178'MB/sec 2024/11/23-04:06:05.762538 140025121105472 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243951.sst): old prealloc_size = 147639500, real_size = 33156560 2024/11/23-04:06:05.763102 140025121105472 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243951.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929171 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.45 avg-zval = 33.52 2024/11/23-04:06:05.763889 140025121105472 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243951.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:06:06.711514 140025121105472 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243951.sst): second pass time = 1.51's, 71.700'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.266'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3283.505'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 291.549'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31426.426'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.700'MB/sec zip pipeline throughput = 102.818'MB/sec entries = 929171 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.45 avg-zval = 33.52 usrkeys = 929171 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.45 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929171 bytes = 3252112 } seq expand size = 6504197 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4441 value = 3.4741 dict = 1.00 all = 3.6020 } Zip/UnZip{ index = 0.0957 value = 0.2878 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 0.916287 GB avg = 0.123 KB (by entry num) total key len = 0.208233 GB avg = 0.028 KB total ukey len = 0.148738 GB avg = 0.020 KB total ukey num = 0.007436887 Billion total entry num = 0.007436887 Billion write speed all = 0.016244739 MB/sec (with seq num) write speed all = 0.015385276 MB/sec (without seq num) 2024/11/23-04:06:06.744537 140025121105472 [compaction_job.cc:1948] [default] [JOB 18936] Generated table #243951: 929171 keys, 37261816 bytes, temperature: kWarm 2024/11/23-04:06:06.744601 140025121105472 EVENT_LOG_v1 {"time_micros": 1732305966744568, "cf_name": "default", "job": 18936, "event": "table_file_creation", "file_number": 243951, "file_size": 37261816, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 3775662187, "table_properties": {"data_size": 31144768, "index_size": 1779328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26016788, "raw_average_key_size": 28, "raw_value_size": 108201006, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929171, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305963, "oldest_key_time": 0, "file_creation_time": 1732305963, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243951, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:06:06.781588 140025121105472 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243952.sst) /dev/shm/ToplingTemp/Topling-403519-bYX4po 2024/11/23-04:06:07.584470 140025121105472 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243952.sst): first pass time = 0.80's, 167.178'MB/sec 2024/11/23-04:06:09.151280 140025121105472 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243952.sst): old prealloc_size = 147639500, real_size = 33409488 2024/11/23-04:06:09.151840 140025121105472 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243952.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935069 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0314 GB avg-val = 115.54 avg-zval = 33.56 2024/11/23-04:06:09.152635 140025121105472 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243952.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:06:10.069117 140025121105472 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243952.sst): second pass time = 1.57's, 68.981'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.020'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3312.343'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.296'MB/sec write SST data time = 0.13's, 293.812'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35937.654'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.981'MB/sec zip pipeline throughput = 119.041'MB/sec entries = 935069 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.54 avg-zval = 33.56 usrkeys = 935069 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.54 avg-zval = 33.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935069 bytes = 3389648 } seq expand size = 6545483 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4428 value = 3.4423 dict = 1.00 all = 3.5650 } Zip/UnZip{ index = 0.0958 value = 0.2905 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 1.030868 GB avg = 0.123 KB (by entry num) total key len = 0.234415 GB avg = 0.028 KB total ukey len = 0.167439 GB avg = 0.020 KB total ukey num = 0.008371956 Billion total entry num = 0.008371956 Billion write speed all = 0.018277310 MB/sec (with seq num) write speed all = 0.017309831 MB/sec (without seq num) 2024/11/23-04:06:10.106210 140025121105472 [compaction_job.cc:1948] [default] [JOB 18936] Generated table #243952: 935069 keys, 37649130 bytes, temperature: kWarm 2024/11/23-04:06:10.106271 140025121105472 EVENT_LOG_v1 {"time_micros": 1732305970106240, "cf_name": "default", "job": 18936, "event": "table_file_creation", "file_number": 243952, "file_size": 37649130, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3775662850, "table_properties": {"data_size": 31384704, "index_size": 1790848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26181932, "raw_average_key_size": 28, "raw_value_size": 108035870, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935069, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305966, "oldest_key_time": 0, "file_creation_time": 1732305966, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243952, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:06:10.149788 140025121105472 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243953.sst) /dev/shm/ToplingTemp/Topling-403519-RFtcVp 2024/11/23-04:06:10.898600 140025121105472 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243953.sst): first pass time = 0.75's, 179.249'MB/sec 2024/11/23-04:06:12.512877 140025121105472 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243953.sst): old prealloc_size = 147639500, real_size = 32844976 2024/11/23-04:06:12.513423 140025121105472 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243953.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924847 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.12 avg-zval = 33.34 2024/11/23-04:06:12.514220 140025121105472 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243953.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-04:06:13.437866 140025121105472 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243953.sst): second pass time = 1.61's, 67.150'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.058'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3397.770'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.13's, 287.806'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 41100.084'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.150'MB/sec zip pipeline throughput = 134.365'MB/sec entries = 924847 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.12 avg-zval = 33.34 usrkeys = 924847 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.12 avg-zval = 33.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924847 bytes = 3352592 } seq expand size = 6473929 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.3768 value = 3.5134 dict = 1.00 all = 3.6223 } Zip/UnZip{ index = 0.0964 value = 0.2846 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 1.145664 GB avg = 0.123 KB (by entry num) total key len = 0.260310 GB avg = 0.028 KB total ukey len = 0.185936 GB avg = 0.020 KB total ukey num = 0.009296803 Billion total entry num = 0.009296803 Billion write speed all = 0.020308647 MB/sec (with seq num) write speed all = 0.019234343 MB/sec (without seq num) 2024/11/23-04:06:13.477776 140025121105472 [compaction_job.cc:1948] [default] [JOB 18936] Generated table #243953: 924847 keys, 37053026 bytes, temperature: kWarm 2024/11/23-04:06:13.477835 140025121105472 EVENT_LOG_v1 {"time_micros": 1732305973477806, "cf_name": "default", "job": 18936, "event": "table_file_creation", "file_number": 243953, "file_size": 37053026, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 3775663164, "table_properties": {"data_size": 30831072, "index_size": 1782528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25895716, "raw_average_key_size": 28, "raw_value_size": 108322070, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924847, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305970, "oldest_key_time": 0, "file_creation_time": 1732305970, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243953, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:06:13.512993 140025121105472 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243954.sst) /dev/shm/ToplingTemp/Topling-403519-b6qSTo 2024/11/23-04:06:13.748641 140025121105472 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243954.sst): first pass time = 0.24's, 246.057'MB/sec 2024/11/23-04:06:14.301964 140025121105472 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243954.sst): old prealloc_size = 147639500, real_size = 14347728 2024/11/23-04:06:14.302234 140025121105472 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243954.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 405193 prefix = 4 raw-key = 0.0065 GB zip-key = 0.0008 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0466 GB zip-val = 0.0135 GB avg-val = 115.08 avg-zval = 33.24 2024/11/23-04:06:14.302592 140025121105472 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243954.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0010 GB, waited 0.000 sec, Key+Value = 0.058 GB 2024/11/23-04:06:14.719747 140025121105472 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00/243954.sst): second pass time = 0.55's, 84.371'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.42's, 38.396'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3110.536'MB/sec (index lex order gen) rebuild zvType time = 0.36's, 0.280'MB/sec write SST data time = 0.06's, 284.608'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.475'MB, throughput = 14734.634'MB/sec zip my value time = 0.55's, unzip length = 0.047'GB zip my value throughput = 84.371'MB/sec zip pipeline throughput = 84.371'MB/sec entries = 405193 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.08 avg-zval = 33.24 usrkeys = 405193 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.08 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 405193 bytes = 1316896 } seq expand size = 2836351 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0081 GB value = 0.0466 GB dict = 0.47 MB all = 0.0580 GB } __ZipSize{ index = 0.0008 GB value = 0.0135 GB dict = 0.47 MB all = 0.0160 GB } UnZip/Zip{ index = 10.4337 value = 3.4618 dict = 1.00 all = 3.6142 } Zip/UnZip{ index = 0.0958 value = 0.2889 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 1.195129 GB avg = 0.123 KB (by entry num) total key len = 0.271656 GB avg = 0.028 KB total ukey len = 0.194040 GB avg = 0.020 KB total ukey num = 0.009701996 Billion total entry num = 0.009701996 Billion write speed all = 0.021186636 MB/sec (with seq num) write speed all = 0.020065530 MB/sec (without seq num) 2024/11/23-04:06:14.736811 140025121105472 [compaction_job.cc:1948] [default] [JOB 18936] Generated table #243954: 405193 keys, 16040687 bytes, temperature: kWarm 2024/11/23-04:06:14.736871 140025121105472 EVENT_LOG_v1 {"time_micros": 1732305974736839, "cf_name": "default", "job": 18936, "event": "table_file_creation", "file_number": 243954, "file_size": 16040687, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3722378739, "largest_seqno": 3775659507, "table_properties": {"data_size": 13469568, "index_size": 776704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 11345404, "raw_average_key_size": 28, "raw_value_size": 46628918, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 405193, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732305973, "oldest_key_time": 0, "file_creation_time": 1732305973, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 243954, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-04:06:14.755099 140025121105472 [compaction_job.cc:703] job-18936: subcompact[0], size: 0.388150 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-18936/att-00 [243944,243945,243946,243947,243948,243949,243950,243951,243952,243953,243954] 2024/11/23-04:06:14.826242 140025121105472 (Original Log Time 2024/11/23-04:05:37.239328) EVENT_LOG_v1 {"time_micros": 1732305937239291, "job": 18936, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [243894], "files_L3": [243940, 243941, 243867, 243868, 243869, 243821, 243822, 243823, 243824, 243825, 243826, 243827], "score": 0.990763, "input_data_size": 411551313, "oldest_snapshot_seqno": 3775993667} 2024/11/23-04:06:14.826245 140025121105472 (Original Log Time 2024/11/23-04:06:14.825635) [compaction_job.cc:2022] [default] [JOB 18936] Compacted 1@2 + 12@3 files to L3 => 388150309 bytes 2024/11/23-04:06:14.826247 140025121105472 (Original Log Time 2024/11/23-04:06:14.826179) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 11.0 rd, 10.3 wr, level 3, files in(1, 12) out(11 +0 blob) MB in(27.1, 365.4 +0.0 blob) out(370.2 +0.0 blob), read-write-amplify(28.1) write-amplify(13.7) OK, records in: 9764770, records dropped: 62774 output_compression: Snappy 2024/11/23-04:06:14.826253 140025121105472 (Original Log Time 2024/11/23-04:06:14.826223) EVENT_LOG_v1 {"time_micros": 1732305974826202, "cf": "default", "job": 18936, "event": "compaction_finished", "compaction_time_micros": 37515778, "compaction_time_cpu_micros": 16370493, "output_level": 3, "num_output_files": 11, "total_output_size": 388150309, "num_input_records": 9764770, "num_output_records": 9701996, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 244025482, "file_range_sync_nanos": 0, "file_fsync_nanos": 9701, "file_prepare_write_nanos": 27431, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/23-04:06:14.956278 140025121105472 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732305937,"executesMs":37715,"compactionJobId":18936,"attempt":0,"compactionInputRawBytes":1409066132,"compactionInputZipBytes":411551313,"compactionOutputRawBytes":0,"compactionOutputZipBytes":388150309,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-41cb50358efe9ec13668076822e2de8e-50c16262dcc453a0-00"}}, response = 2024/11/23-04:06:14.956359 140025121105472 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-18936/att-00: olev 3, work 37.716 s, result 6.490 ms, install 0.648 ms, input{raw 1.312 GiB zip 392.486 MiB}