2024/11/23-03:03:46.424805 140024944858688 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00: fork to child time = 0.009144 sec 2024/11/23-03:03:46.425973 140024944858688 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00 2024/11/23-03:03:46.432560 140024944858688 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00 2024/11/23-03:03:46.433704 140024944858688 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-03:03:46.433997 140024944858688 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-03:03:46.482473 140024944858688 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-17955/att-00/MANIFEST-206347 2024/11/23-03:03:46.516496 140024944858688 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-17955/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 241909, last_sequence is 3749226733, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 241722 2024/11/23-03:03:46.516502 140024944858688 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-03:03:46.516622 140024944858688 [version_set.cc:5721] Creating manifest 241909 2024/11/23-03:03:46.552591 140024944858688 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-03:03:46.574241 140024944858688 [compaction_job.cc:2437] [default] [JOB 17955] Compacting 1@2 + 11@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-03:03:46.574256 140024944858688 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 13409 Base level 2, inputs: [241870(27MB)], [241816(35MB) 241817(35MB) 241818(35MB) 241819(36MB) 241820(37MB) 241821(35MB) 241822(35MB) 241823(22MB) 241779(35MB) 241780(35MB) 241882(36MB)] 2024/11/23-03:03:46.580305 140024944858688 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-03:03:46.580338 140024944858688 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-03:03:46.580356 140024944858688 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-03:03:46.580361 140024944858688 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-03:03:46.580371 140024944858688 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-03:03:47.020681 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241910.sst) /dev/shm/ToplingTemp/Topling-389187-VTnRVF 2024/11/23-03:03:47.585024 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241910.sst): first pass time = 0.56's, 237.847'MB/sec 2024/11/23-03:03:49.393411 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241910.sst): old prealloc_size = 147639500, real_size = 32911744 2024/11/23-03:03:49.393975 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241910.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927650 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.69 avg-zval = 33.33 2024/11/23-03:03:49.394726 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241910.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:03:50.699822 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241910.sst): second pass time = 1.81's, 59.894'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.31's, 28.244'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3237.593'MB/sec (index lex order gen) rebuild zvType time = 1.07's, 0.216'MB/sec write SST data time = 0.23's, 157.734'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33315.140'MB/sec zip my value time = 1.81's, unzip length = 0.108'GB zip my value throughput = 59.894'MB/sec zip pipeline throughput = 59.894'MB/sec entries = 927650 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.69 avg-zval = 33.33 usrkeys = 927650 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.69 avg-zval = 33.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927650 bytes = 3130832 } seq expand size = 6493550 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5124 value = 3.5014 dict = 1.00 all = 3.6376 } Zip/UnZip{ index = 0.0951 value = 0.2856 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 0.114737 GB avg = 0.124 KB (by entry num) total key len = 0.025974 GB avg = 0.028 KB total ukey len = 0.018553 GB avg = 0.020 KB total ukey num = 0.000927650 Billion total entry num = 0.000927650 Billion write speed all = 0.002148672 MB/sec (with seq num) write speed all = 0.002035349 MB/sec (without seq num) 2024/11/23-03:03:50.735333 140024944858688 [compaction_job.cc:1948] [default] [JOB 17955] Generated table #241910: 927650 keys, 36896938 bytes, temperature: kWarm 2024/11/23-03:03:50.735534 140024944858688 EVENT_LOG_v1 {"time_micros": 1732302230735376, "cf_name": "default", "job": 17955, "event": "table_file_creation", "file_number": 241910, "file_size": 36896938, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3748828123, "table_properties": {"data_size": 30914800, "index_size": 1764864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25974200, "raw_average_key_size": 28, "raw_value_size": 108243600, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927650, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302227, "oldest_key_time": 0, "file_creation_time": 1732302227, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241910, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:03:50.783037 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241911.sst) /dev/shm/ToplingTemp/Topling-389187-EdPSrH 2024/11/23-03:03:51.808703 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241911.sst): first pass time = 1.03's, 130.865'MB/sec 2024/11/23-03:03:53.670680 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241911.sst): old prealloc_size = 147639500, real_size = 33251792 2024/11/23-03:03:53.671292 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241911.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927245 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.75 avg-zval = 33.71 2024/11/23-03:03:53.672139 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241911.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:03:54.971060 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241911.sst): second pass time = 1.86's, 58.169'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.30's, 28.813'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2971.182'MB/sec (index lex order gen) rebuild zvType time = 1.05's, 0.221'MB/sec write SST data time = 0.25's, 149.410'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30653.119'MB/sec zip my value time = 1.86's, unzip length = 0.108'GB zip my value throughput = 58.169'MB/sec zip pipeline throughput = 99.342'MB/sec entries = 927245 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.75 avg-zval = 33.71 usrkeys = 927245 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.75 avg-zval = 33.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927245 bytes = 3361280 } seq expand size = 6490715 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5113 value = 3.4635 dict = 1.00 all = 3.5823 } Zip/UnZip{ index = 0.0951 value = 0.2887 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 0.229483 GB avg = 0.124 KB (by entry num) total key len = 0.051937 GB avg = 0.028 KB total ukey len = 0.037098 GB avg = 0.020 KB total ukey num = 0.001854895 Billion total entry num = 0.001854895 Billion write speed all = 0.004297019 MB/sec (with seq num) write speed all = 0.004070438 MB/sec (without seq num) 2024/11/23-03:03:55.008466 140024944858688 [compaction_job.cc:1948] [default] [JOB 17955] Generated table #241911: 927245 keys, 37467120 bytes, temperature: kWarm 2024/11/23-03:03:55.008559 140024944858688 EVENT_LOG_v1 {"time_micros": 1732302235008503, "cf_name": "default", "job": 17955, "event": "table_file_creation", "file_number": 241911, "file_size": 37467120, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3748826627, "table_properties": {"data_size": 31255488, "index_size": 1764288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25962860, "raw_average_key_size": 28, "raw_value_size": 108254878, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927245, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302230, "oldest_key_time": 0, "file_creation_time": 1732302230, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241911, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:03:55.086392 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241912.sst) /dev/shm/ToplingTemp/Topling-389187-lfapVG 2024/11/23-03:03:55.920772 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241912.sst): first pass time = 0.83's, 160.867'MB/sec 2024/11/23-03:03:57.688463 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241912.sst): old prealloc_size = 147639500, real_size = 33248720 2024/11/23-03:03:57.689014 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241912.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934365 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 115.65 avg-zval = 33.41 2024/11/23-03:03:57.689772 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241912.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:03:58.736804 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241912.sst): second pass time = 1.77's, 61.167'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 35.758'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3462.286'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.255'MB/sec write SST data time = 0.13's, 288.137'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 40591.407'MB/sec zip my value time = 1.77's, unzip length = 0.108'GB zip my value throughput = 61.167'MB/sec zip pipeline throughput = 61.167'MB/sec entries = 934365 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.65 avg-zval = 33.41 usrkeys = 934365 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.65 avg-zval = 33.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934365 bytes = 3387088 } seq expand size = 6540555 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.3756 value = 3.4618 dict = 1.00 all = 3.5805 } Zip/UnZip{ index = 0.0964 value = 0.2889 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 0.344079 GB avg = 0.123 KB (by entry num) total key len = 0.078099 GB avg = 0.028 KB total ukey len = 0.055785 GB avg = 0.020 KB total ukey num = 0.002789260 Billion total entry num = 0.002789260 Billion write speed all = 0.006445897 MB/sec (with seq num) write speed all = 0.006105201 MB/sec (without seq num) 2024/11/23-03:03:58.768772 140024944858688 [compaction_job.cc:1948] [default] [JOB 17955] Generated table #241912: 934365 keys, 37486130 bytes, temperature: kWarm 2024/11/23-03:03:58.768855 140024944858688 EVENT_LOG_v1 {"time_micros": 1732302238768803, "cf_name": "default", "job": 17955, "event": "table_file_creation", "file_number": 241912, "file_size": 37486130, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3339428666, "largest_seqno": 3748828219, "table_properties": {"data_size": 31213888, "index_size": 1801088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26162220, "raw_average_key_size": 28, "raw_value_size": 108055574, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934365, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302235, "oldest_key_time": 0, "file_creation_time": 1732302235, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241912, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:03:58.804975 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241913.sst) /dev/shm/ToplingTemp/Topling-389187-53vCqG 2024/11/23-03:03:59.599468 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241913.sst): first pass time = 0.79's, 168.944'MB/sec 2024/11/23-03:04:00.968366 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241913.sst): old prealloc_size = 147639500, real_size = 33976928 2024/11/23-03:04:00.968953 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241913.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931092 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0320 GB avg-val = 116.15 avg-zval = 34.34 2024/11/23-03:04:00.969737 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241913.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:04:01.894668 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241913.sst): second pass time = 1.37's, 79.084'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 41.244'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3121.102'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.13's, 297.913'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34142.369'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.084'MB/sec zip pipeline throughput = 79.084'MB/sec entries = 931092 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.15 avg-zval = 34.34 usrkeys = 931092 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 34.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931092 bytes = 3375216 } seq expand size = 6517644 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5099 value = 3.3825 dict = 1.00 all = 3.5132 } Zip/UnZip{ index = 0.0951 value = 0.2956 dict = 1.00 all = 0.2846 } ---------------------------- total value len = 0.458744 GB avg = 0.123 KB (by entry num) total key len = 0.104170 GB avg = 0.028 KB total ukey len = 0.074407 GB avg = 0.020 KB total ukey num = 0.003720352 Billion total entry num = 0.003720352 Billion write speed all = 0.008594257 MB/sec (with seq num) write speed all = 0.008139855 MB/sec (without seq num) 2024/11/23-03:04:01.927947 140024944858688 [compaction_job.cc:1948] [default] [JOB 17955] Generated table #241913: 931092 keys, 38203352 bytes, temperature: kWarm 2024/11/23-03:04:01.928010 140024944858688 EVENT_LOG_v1 {"time_micros": 1732302241927977, "cf_name": "default", "job": 17955, "event": "table_file_creation", "file_number": 241913, "file_size": 38203352, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3748827566, "table_properties": {"data_size": 31972112, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070576, "raw_average_key_size": 28, "raw_value_size": 108147168, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931092, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302238, "oldest_key_time": 0, "file_creation_time": 1732302238, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241913, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:04:01.969377 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241914.sst) /dev/shm/ToplingTemp/Topling-389187-nsAuRG 2024/11/23-03:04:02.843379 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241914.sst): first pass time = 0.87's, 153.573'MB/sec 2024/11/23-03:04:04.179109 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241914.sst): old prealloc_size = 147639500, real_size = 34677136 2024/11/23-03:04:04.179676 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241914.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930195 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0327 GB avg-val = 116.29 avg-zval = 35.11 2024/11/23-03:04:04.180356 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241914.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:04:05.067190 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241914.sst): second pass time = 1.34's, 81.025'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.226'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3277.652'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.308'MB/sec write SST data time = 0.13's, 291.856'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33963.866'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.025'MB/sec zip pipeline throughput = 81.025'MB/sec entries = 930195 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.29 avg-zval = 35.11 usrkeys = 930195 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.91 avg-zval = 35.11 TagRS{ kinds = RS_Key0_TagN bytes = 145440 } TagArray{ size = 747198 bytes = 2708608 } seq expand size = 5230386 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4447 value = 3.3117 dict = 1.00 all = 3.4964 } Zip/UnZip{ index = 0.0957 value = 0.3020 dict = 1.00 all = 0.2860 } ---------------------------- total value len = 0.572146 GB avg = 0.123 KB (by entry num) total key len = 0.130215 GB avg = 0.028 KB total ukey len = 0.093011 GB avg = 0.020 KB total ukey num = 0.004650547 Billion total entry num = 0.004650547 Billion write speed all = 0.010722756 MB/sec (with seq num) write speed all = 0.010154768 MB/sec (without seq num) 2024/11/23-03:04:05.101084 140024944858688 [compaction_job.cc:1948] [default] [JOB 17955] Generated table #241914: 930195 keys, 38387178 bytes, temperature: kWarm 2024/11/23-03:04:05.101148 140024944858688 EVENT_LOG_v1 {"time_micros": 1732302245101114, "cf_name": "default", "job": 17955, "event": "table_file_creation", "file_number": 241914, "file_size": 38387178, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3748827384, "table_properties": {"data_size": 32663232, "index_size": 1781184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26045460, "raw_average_key_size": 28, "raw_value_size": 108172294, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930195, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302241, "oldest_key_time": 0, "file_creation_time": 1732302241, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241914, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:04:05.137678 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241915.sst) /dev/shm/ToplingTemp/Topling-389187-dc6sSH 2024/11/23-03:04:05.999148 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241915.sst): first pass time = 0.86's, 155.808'MB/sec 2024/11/23-03:04:07.346078 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241915.sst): old prealloc_size = 147639500, real_size = 32547552 2024/11/23-03:04:07.346624 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241915.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927526 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.71 avg-zval = 32.94 2024/11/23-03:04:07.347403 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241915.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:04:08.246118 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241915.sst): second pass time = 1.35's, 80.410'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.719'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3355.928'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.13's, 286.162'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32242.741'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.410'MB/sec zip pipeline throughput = 80.410'MB/sec entries = 927526 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.71 avg-zval = 32.94 usrkeys = 927526 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.71 avg-zval = 32.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927526 bytes = 3246352 } seq expand size = 6492682 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5137 value = 3.5432 dict = 1.00 all = 3.6623 } Zip/UnZip{ index = 0.0951 value = 0.2822 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 0.686886 GB avg = 0.123 KB (by entry num) total key len = 0.156186 GB avg = 0.028 KB total ukey len = 0.111561 GB avg = 0.020 KB total ukey num = 0.005578073 Billion total entry num = 0.005578073 Billion write speed all = 0.012870320 MB/sec (with seq num) write speed all = 0.012189082 MB/sec (without seq num) 2024/11/23-03:04:08.277972 140024944858688 [compaction_job.cc:1948] [default] [JOB 17955] Generated table #241915: 927526 keys, 36648246 bytes, temperature: kWarm 2024/11/23-03:04:08.278035 140024944858688 EVENT_LOG_v1 {"time_micros": 1732302248278000, "cf_name": "default", "job": 17955, "event": "table_file_creation", "file_number": 241915, "file_size": 36648246, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 3748827980, "table_properties": {"data_size": 30551056, "index_size": 1764416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25970728, "raw_average_key_size": 28, "raw_value_size": 108247132, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927526, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302245, "oldest_key_time": 0, "file_creation_time": 1732302245, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241915, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:04:08.316128 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241916.sst) /dev/shm/ToplingTemp/Topling-389187-oN740H 2024/11/23-03:04:09.220539 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241916.sst): first pass time = 0.90's, 148.409'MB/sec 2024/11/23-03:04:10.558560 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241916.sst): old prealloc_size = 147639500, real_size = 33417120 2024/11/23-03:04:10.559093 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241916.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926224 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.91 avg-zval = 33.93 2024/11/23-03:04:10.559864 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241916.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:04:11.507506 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241916.sst): second pass time = 1.34's, 80.971'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.532'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3456.643'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 291.836'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34448.122'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.971'MB/sec zip pipeline throughput = 80.971'MB/sec entries = 926224 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 33.93 usrkeys = 926224 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.91 avg-zval = 33.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926224 bytes = 3241792 } seq expand size = 6483568 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5138 value = 3.4459 dict = 1.00 all = 3.5779 } Zip/UnZip{ index = 0.0951 value = 0.2902 dict = 1.00 all = 0.2795 } ---------------------------- total value len = 0.801653 GB avg = 0.123 KB (by entry num) total key len = 0.182120 GB avg = 0.028 KB total ukey len = 0.130086 GB avg = 0.020 KB total ukey num = 0.006504297 Billion total entry num = 0.006504297 Billion write speed all = 0.015017517 MB/sec (with seq num) write speed all = 0.014223201 MB/sec (without seq num) 2024/11/23-03:04:11.539523 140024944858688 [compaction_job.cc:1948] [default] [JOB 17955] Generated table #241916: 926224 keys, 37513530 bytes, temperature: kWarm 2024/11/23-03:04:11.539582 140024944858688 EVENT_LOG_v1 {"time_micros": 1732302251539551, "cf_name": "default", "job": 17955, "event": "table_file_creation", "file_number": 241916, "file_size": 37513530, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3604582272, "largest_seqno": 3748827108, "table_properties": {"data_size": 31423440, "index_size": 1761920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25934272, "raw_average_key_size": 28, "raw_value_size": 108283588, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926224, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302248, "oldest_key_time": 0, "file_creation_time": 1732302248, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241916, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:04:11.579393 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241917.sst) /dev/shm/ToplingTemp/Topling-389187-eNm6NH 2024/11/23-03:04:12.641645 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241917.sst): first pass time = 1.06's, 126.356'MB/sec 2024/11/23-03:04:13.972452 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241917.sst): old prealloc_size = 147639500, real_size = 33334080 2024/11/23-03:04:13.973030 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241917.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927148 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.76 avg-zval = 33.80 2024/11/23-03:04:13.973797 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241917.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:04:14.920394 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241917.sst): second pass time = 1.33's, 81.391'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.490'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3160.253'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 295.144'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32461.063'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.391'MB/sec zip pipeline throughput = 81.391'MB/sec entries = 927148 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 33.80 usrkeys = 927148 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 33.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927148 bytes = 3245040 } seq expand size = 6490036 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5121 value = 3.4545 dict = 1.00 all = 3.5855 } Zip/UnZip{ index = 0.0951 value = 0.2895 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 0.916401 GB avg = 0.123 KB (by entry num) total key len = 0.208080 GB avg = 0.028 KB total ukey len = 0.148629 GB avg = 0.020 KB total ukey num = 0.007431445 Billion total entry num = 0.007431445 Billion write speed all = 0.017164558 MB/sec (with seq num) write speed all = 0.016257065 MB/sec (without seq num) 2024/11/23-03:04:14.953597 140024944858688 [compaction_job.cc:1948] [default] [JOB 17955] Generated table #241917: 927148 keys, 37434024 bytes, temperature: kWarm 2024/11/23-03:04:14.953661 140024944858688 EVENT_LOG_v1 {"time_micros": 1732302254953627, "cf_name": "default", "job": 17955, "event": "table_file_creation", "file_number": 241917, "file_size": 37434024, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 3748827124, "table_properties": {"data_size": 31338160, "index_size": 1763968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960144, "raw_average_key_size": 28, "raw_value_size": 108257704, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927148, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302251, "oldest_key_time": 0, "file_creation_time": 1732302251, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241917, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:04:14.992272 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241918.sst) /dev/shm/ToplingTemp/Topling-389187-od7ooJ 2024/11/23-03:04:15.854341 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241918.sst): first pass time = 0.86's, 155.699'MB/sec 2024/11/23-03:04:17.198503 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241918.sst): old prealloc_size = 147639500, real_size = 33431280 2024/11/23-03:04:17.199083 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241918.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927107 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.77 avg-zval = 33.91 2024/11/23-03:04:17.199886 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241918.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:04:18.155464 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241918.sst): second pass time = 1.34's, 80.579'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 39.340'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3166.406'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.13's, 292.605'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31172.225'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.579'MB/sec zip pipeline throughput = 80.579'MB/sec entries = 927107 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 33.91 usrkeys = 927107 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.77 avg-zval = 33.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927107 bytes = 3360784 } seq expand size = 6489749 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5124 value = 3.4438 dict = 1.00 all = 3.5652 } Zip/UnZip{ index = 0.0951 value = 0.2904 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 1.031150 GB avg = 0.123 KB (by entry num) total key len = 0.234039 GB avg = 0.028 KB total ukey len = 0.167171 GB avg = 0.020 KB total ukey num = 0.008358552 Billion total entry num = 0.008358552 Billion write speed all = 0.019311424 MB/sec (with seq num) write speed all = 0.018290767 MB/sec (without seq num) 2024/11/23-03:04:18.187857 140024944858688 [compaction_job.cc:1948] [default] [JOB 17955] Generated table #241918: 927107 keys, 37646714 bytes, temperature: kWarm 2024/11/23-03:04:18.187924 140024944858688 EVENT_LOG_v1 {"time_micros": 1732302258187889, "cf_name": "default", "job": 17955, "event": "table_file_creation", "file_number": 241918, "file_size": 37646714, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268533, "largest_seqno": 3748828150, "table_properties": {"data_size": 31435488, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958996, "raw_average_key_size": 28, "raw_value_size": 108258826, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927107, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302254, "oldest_key_time": 0, "file_creation_time": 1732302254, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241918, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:04:18.229156 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241919.sst) /dev/shm/ToplingTemp/Topling-389187-qYdZbI 2024/11/23-03:04:19.076463 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241919.sst): first pass time = 0.85's, 158.412'MB/sec 2024/11/23-03:04:20.400221 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241919.sst): old prealloc_size = 147639500, real_size = 33456992 2024/11/23-03:04:20.400802 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241919.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932631 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0314 GB avg-val = 115.91 avg-zval = 33.71 2024/11/23-03:04:20.401563 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241919.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-03:04:21.328098 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241919.sst): second pass time = 1.32's, 81.726'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.492'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3183.220'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.13's, 294.645'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32530.830'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.726'MB/sec zip pipeline throughput = 81.726'MB/sec entries = 932631 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.91 avg-zval = 33.71 usrkeys = 932631 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.91 avg-zval = 33.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932631 bytes = 3264224 } seq expand size = 6528417 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4443 value = 3.4387 dict = 1.00 all = 3.5723 } Zip/UnZip{ index = 0.0957 value = 0.2908 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 1.145782 GB avg = 0.123 KB (by entry num) total key len = 0.260153 GB avg = 0.028 KB total ukey len = 0.185824 GB avg = 0.020 KB total ukey num = 0.009291183 Billion total entry num = 0.009291183 Billion write speed all = 0.021458689 MB/sec (with seq num) write speed all = 0.020324204 MB/sec (without seq num) 2024/11/23-03:04:21.362062 140024944858688 [compaction_job.cc:1948] [default] [JOB 17955] Generated table #241919: 932631 keys, 37571748 bytes, temperature: kWarm 2024/11/23-03:04:21.362121 140024944858688 EVENT_LOG_v1 {"time_micros": 1732302261362090, "cf_name": "default", "job": 17955, "event": "table_file_creation", "file_number": 241919, "file_size": 37571748, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 3748827512, "table_properties": {"data_size": 31437712, "index_size": 1785920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26113668, "raw_average_key_size": 28, "raw_value_size": 108104186, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932631, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302258, "oldest_key_time": 0, "file_creation_time": 1732302258, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241919, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:04:21.399531 140024944858688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241920.sst) /dev/shm/ToplingTemp/Topling-389187-43EPRH 2024/11/23-03:04:21.833889 140024944858688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241920.sst): first pass time = 0.43's, 250.521'MB/sec 2024/11/23-03:04:22.918222 140024944858688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241920.sst): old prealloc_size = 147639500, real_size = 27283584 2024/11/23-03:04:22.918657 140024944858688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241920.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 748580 prefix = 4 raw-key = 0.0120 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0878 GB zip-val = 0.0257 GB avg-val = 117.35 avg-zval = 34.29 2024/11/23-03:04:22.919331 140024944858688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241920.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0019 GB, waited 0.000 sec, Key+Value = 0.109 GB 2024/11/23-03:04:23.688076 140024944858688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00/241920.sst): second pass time = 1.08's, 81.062'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 39.870'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3443.219'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.281'MB/sec write SST data time = 0.10's, 295.075'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.879'MB, throughput = 28365.056'MB/sec zip my value time = 1.08's, unzip length = 0.088'GB zip my value throughput = 81.062'MB/sec zip pipeline throughput = 81.062'MB/sec entries = 748580 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.35 avg-zval = 34.29 usrkeys = 748580 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.35 avg-zval = 34.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 748580 bytes = 2713616 } seq expand size = 5240060 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0150 GB value = 0.0878 GB dict = 0.88 MB all = 0.1088 GB } __ZipSize{ index = 0.0014 GB value = 0.0257 GB dict = 0.88 MB all = 0.0307 GB } UnZip/Zip{ index = 10.5109 value = 3.4219 dict = 1.00 all = 3.5449 } Zip/UnZip{ index = 0.0951 value = 0.2922 dict = 1.00 all = 0.2821 } ---------------------------- total value len = 1.238869 GB avg = 0.123 KB (by entry num) total key len = 0.281113 GB avg = 0.028 KB total ukey len = 0.200795 GB avg = 0.020 KB total ukey num = 0.010039763 Billion total entry num = 0.010039763 Billion write speed all = 0.023198549 MB/sec (with seq num) write speed all = 0.021972704 MB/sec (without seq num) 2024/11/23-03:04:23.716689 140024944858688 [compaction_job.cc:1948] [default] [JOB 17955] Generated table #241920: 748580 keys, 30693940 bytes, temperature: kWarm 2024/11/23-03:04:23.716757 140024944858688 EVENT_LOG_v1 {"time_micros": 1732302263716723, "cf_name": "default", "job": 17955, "event": "table_file_creation", "file_number": 241920, "file_size": 30693940, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3748827667, "table_properties": {"data_size": 25671856, "index_size": 1424384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 20960240, "raw_average_key_size": 28, "raw_value_size": 87846972, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 748580, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732302261, "oldest_key_time": 0, "file_creation_time": 1732302261, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241920, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-03:04:23.746820 140024944858688 [compaction_job.cc:703] job-17955: subcompact[0], size: 0.405949 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17955/att-00 [241910,241911,241912,241913,241914,241915,241916,241917,241918,241919,241920] 2024/11/23-03:04:23.819013 140024944858688 (Original Log Time 2024/11/23-03:03:46.574297) EVENT_LOG_v1 {"time_micros": 1732302226574267, "job": 17955, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [241870], "files_L3": [241816, 241817, 241818, 241819, 241820, 241821, 241822, 241823, 241779, 241780, 241882], "score": 0.990856, "input_data_size": 427995433, "oldest_snapshot_seqno": 3749226677} 2024/11/23-03:04:23.819017 140024944858688 (Original Log Time 2024/11/23-03:04:23.818406) [compaction_job.cc:2022] [default] [JOB 17955] Compacted 1@2 + 11@3 files to L3 => 405948920 bytes 2024/11/23-03:04:23.819018 140024944858688 (Original Log Time 2024/11/23-03:04:23.818948) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.02, MB/sec: 11.5 rd, 10.9 wr, level 3, files in(1, 11) out(11 +0 blob) MB in(27.1, 381.1 +0.0 blob) out(387.1 +0.0 blob), read-write-amplify(29.4) write-amplify(14.3) OK, records in: 10085911, records dropped: 46148 output_compression: Snappy 2024/11/23-03:04:23.819025 140024944858688 (Original Log Time 2024/11/23-03:04:23.818993) EVENT_LOG_v1 {"time_micros": 1732302263818971, "cf": "default", "job": 17955, "event": "compaction_finished", "compaction_time_micros": 37172532, "compaction_time_cpu_micros": 16876058, "output_level": 3, "num_output_files": 11, "total_output_size": 405948920, "num_input_records": 10085911, "num_output_records": 10039763, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 268557393, "file_range_sync_nanos": 0, "file_fsync_nanos": 10528, "file_prepare_write_nanos": 25965, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/23-03:04:23.949848 140024944858688 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732302226,"executesMs":37386,"compactionJobId":17955,"attempt":0,"compactionInputRawBytes":1457538302,"compactionInputZipBytes":427995433,"compactionOutputRawBytes":0,"compactionOutputZipBytes":405948920,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-d92fc31a4a78ee7e7c1eb5a063a26497-7f4b6af188c4d253-00"}}, response = 2024/11/23-03:04:23.949915 140024944858688 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-17955/att-00: olev 3, work 37.387 s, result 6.345 ms, install 0.637 ms, input{raw 1.357 GiB zip 408.168 MiB}