2024/11/24-00:01:21.964994 139957634799168 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00: fork to child time = 0.006034 sec 2024/11/24-00:01:21.965528 139957634799168 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00 2024/11/24-00:01:21.972457 139957634799168 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00 2024/11/24-00:01:21.973374 139957634799168 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-00:01:21.973690 139957634799168 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-00:01:22.030272 139957634799168 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38245/att-00/MANIFEST-206347 2024/11/24-00:01:22.057765 139957634799168 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38245/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 285204, last_sequence is 4338157797, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 285088 2024/11/24-00:01:22.057771 139957634799168 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-00:01:22.057858 139957634799168 [version_set.cc:5721] Creating manifest 285204 2024/11/24-00:01:22.092484 139957634799168 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-00:01:22.111436 139957634799168 [compaction_job.cc:2437] [default] [JOB 38245] Compacting 1@2 + 11@3 files to L3, score 0.50, subcompactions 1 : 1 2024/11/24-00:01:22.111449 139957634799168 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 28663 Base level 2, inputs: [285157(27MB)], [285115(35MB) 285116(35MB) 285117(35MB) 285118(35MB) 285119(34MB) 285120(3248KB) 285102(35MB) 285103(35MB) 285104(2141KB) 285105(35MB) 285191(35MB)] 2024/11/24-00:01:22.119118 139957634799168 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:01:22.119145 139957634799168 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:01:22.119166 139957634799168 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:01:22.119171 139957634799168 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:01:22.119181 139957634799168 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:01:22.828551 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285205.sst) /dev/shm/ToplingTemp/Topling-222267-g9inJY 2024/11/24-00:01:23.377657 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285205.sst): first pass time = 0.55's, 244.449'MB/sec 2024/11/24-00:01:24.829010 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285205.sst): old prealloc_size = 147639500, real_size = 33053904 2024/11/24-00:01:24.829554 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285205.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925140 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 117.08 avg-zval = 33.56 2024/11/24-00:01:24.830327 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285205.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:01:25.669274 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285205.sst): second pass time = 1.45's, 74.682'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.071'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3399.082'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.12's, 297.634'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 37142.759'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.682'MB/sec zip pipeline throughput = 74.682'MB/sec entries = 925140 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.08 avg-zval = 33.56 usrkeys = 925140 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.08 avg-zval = 33.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925140 bytes = 3122368 } seq expand size = 6475980 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4446 value = 3.4883 dict = 1.00 all = 3.6244 } Zip/UnZip{ index = 0.0957 value = 0.2867 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 8.963410 GB avg = 0.168 KB (by entry num) total key len = 1.466725 GB avg = 0.028 KB total ukey len = 1.016381 GB avg = 0.015 KB total ukey num = 0.066475575 Billion total entry num = 0.053283304 Billion write speed all = 0.191824021 MB/sec (with seq num) write speed all = 0.183984415 MB/sec (without seq num) 2024/11/24-00:01:25.723506 139957634799168 [compaction_job.cc:1948] [default] [JOB 38245] Generated table #285205: 925140 keys, 37031772 bytes, temperature: kWarm 2024/11/24-00:01:25.723604 139957634799168 EVENT_LOG_v1 {"time_micros": 1732377685723545, "cf_name": "default", "job": 38245, "event": "table_file_creation", "file_number": 285205, "file_size": 37031772, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4259662629, "largest_seqno": 4337854896, "table_properties": {"data_size": 31050944, "index_size": 1771520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903920, "raw_average_key_size": 28, "raw_value_size": 108313924, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925140, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377682, "oldest_key_time": 0, "file_creation_time": 1732377682, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285205, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:01:25.760433 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285206.sst) /dev/shm/ToplingTemp/Topling-222267-Z9VAL0 2024/11/24-00:01:26.704025 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285206.sst): first pass time = 0.94's, 142.249'MB/sec 2024/11/24-00:01:28.177236 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285206.sst): old prealloc_size = 147639500, real_size = 33475760 2024/11/24-00:01:28.177759 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285206.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925999 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.94 avg-zval = 34.00 2024/11/24-00:01:28.178527 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285206.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:01:29.032841 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285206.sst): second pass time = 1.47's, 73.546'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.778'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3540.729'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 298.805'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26439.106'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.546'MB/sec zip pipeline throughput = 74.225'MB/sec entries = 925999 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 34.00 usrkeys = 925999 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.94 avg-zval = 34.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925999 bytes = 3125264 } seq expand size = 6481993 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5112 value = 3.4397 dict = 1.00 all = 3.5833 } Zip/UnZip{ index = 0.0951 value = 0.2907 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 9.078182 GB avg = 0.167 KB (by entry num) total key len = 1.492653 GB avg = 0.028 KB total ukey len = 1.034901 GB avg = 0.015 KB total ukey num = 0.067401574 Billion total entry num = 0.054209303 Billion write speed all = 0.194399652 MB/sec (with seq num) write speed all = 0.186424297 MB/sec (without seq num) 2024/11/24-00:01:29.088322 139957634799168 [compaction_job.cc:1948] [default] [JOB 38245] Generated table #285206: 925999 keys, 37456048 bytes, temperature: kWarm 2024/11/24-00:01:29.088411 139957634799168 EVENT_LOG_v1 {"time_micros": 1732377689088361, "cf_name": "default", "job": 38245, "event": "table_file_creation", "file_number": 285206, "file_size": 37456048, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178503, "largest_seqno": 4337855473, "table_properties": {"data_size": 31482144, "index_size": 1761920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927972, "raw_average_key_size": 28, "raw_value_size": 108289890, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925999, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377685, "oldest_key_time": 0, "file_creation_time": 1732377685, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285206, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:01:29.132350 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285207.sst) /dev/shm/ToplingTemp/Topling-222267-n6CqnY 2024/11/24-00:01:30.098342 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285207.sst): first pass time = 0.97's, 138.950'MB/sec 2024/11/24-00:01:31.434865 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285207.sst): old prealloc_size = 147639500, real_size = 33106336 2024/11/24-00:01:31.435440 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285207.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929814 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.35 avg-zval = 33.45 2024/11/24-00:01:31.436198 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285207.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:01:32.277859 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285207.sst): second pass time = 1.34's, 81.016'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.007'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3202.937'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.12's, 299.037'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30781.465'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.016'MB/sec zip pipeline throughput = 81.016'MB/sec entries = 929814 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.35 avg-zval = 33.45 usrkeys = 929814 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.35 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929814 bytes = 3138144 } seq expand size = 6508698 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5114 value = 3.4780 dict = 1.00 all = 3.6181 } Zip/UnZip{ index = 0.0951 value = 0.2875 dict = 1.00 all = 0.2764 } ---------------------------- total value len = 9.192873 GB avg = 0.167 KB (by entry num) total key len = 1.518688 GB avg = 0.028 KB total ukey len = 1.053497 GB avg = 0.015 KB total ukey num = 0.068331388 Billion total entry num = 0.055139117 Billion write speed all = 0.196975886 MB/sec (with seq num) write speed all = 0.188864219 MB/sec (without seq num) 2024/11/24-00:01:32.331804 139957634799168 [compaction_job.cc:1948] [default] [JOB 38245] Generated table #285207: 929814 keys, 37096606 bytes, temperature: kWarm 2024/11/24-00:01:32.331875 139957634799168 EVENT_LOG_v1 {"time_micros": 1732377692331839, "cf_name": "default", "job": 38245, "event": "table_file_creation", "file_number": 285207, "file_size": 37096606, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178518, "largest_seqno": 4337855192, "table_properties": {"data_size": 31104528, "index_size": 1769152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26034792, "raw_average_key_size": 28, "raw_value_size": 108183064, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929814, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377689, "oldest_key_time": 0, "file_creation_time": 1732377689, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285207, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:01:32.365127 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285208.sst) /dev/shm/ToplingTemp/Topling-222267-QOWsBZ 2024/11/24-00:01:33.293457 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285208.sst): first pass time = 0.93's, 144.587'MB/sec 2024/11/24-00:01:34.619092 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285208.sst): old prealloc_size = 147639500, real_size = 33354368 2024/11/24-00:01:34.619622 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285208.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924984 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.10 avg-zval = 33.91 2024/11/24-00:01:34.620373 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285208.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:01:35.467744 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285208.sst): second pass time = 1.32's, 81.761'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.991'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3466.014'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.319'MB/sec write SST data time = 0.12's, 301.508'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33587.683'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.761'MB/sec zip pipeline throughput = 81.761'MB/sec entries = 924984 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 33.91 usrkeys = 924984 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 33.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924984 bytes = 3121840 } seq expand size = 6474888 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5127 value = 3.4537 dict = 1.00 all = 3.5953 } Zip/UnZip{ index = 0.0951 value = 0.2895 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 9.307666 GB avg = 0.166 KB (by entry num) total key len = 1.544587 GB avg = 0.028 KB total ukey len = 1.071997 GB avg = 0.015 KB total ukey num = 0.069256372 Billion total entry num = 0.056064101 Billion write speed all = 0.199551391 MB/sec (with seq num) write speed all = 0.191304131 MB/sec (without seq num) 2024/11/24-00:01:35.521039 139957634799168 [compaction_job.cc:1948] [default] [JOB 38245] Generated table #285208: 924984 keys, 37331754 bytes, temperature: kWarm 2024/11/24-00:01:35.521117 139957634799168 EVENT_LOG_v1 {"time_micros": 1732377695521076, "cf_name": "default", "job": 38245, "event": "table_file_creation", "file_number": 285208, "file_size": 37331754, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178548, "largest_seqno": 4337855562, "table_properties": {"data_size": 31363184, "index_size": 1759744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25899552, "raw_average_key_size": 28, "raw_value_size": 108318236, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924984, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377692, "oldest_key_time": 0, "file_creation_time": 1732377692, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285208, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:01:35.557256 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285209.sst) /dev/shm/ToplingTemp/Topling-222267-nsF3qY 2024/11/24-00:01:36.578359 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285209.sst): first pass time = 1.02's, 131.449'MB/sec 2024/11/24-00:01:37.899506 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285209.sst): old prealloc_size = 147639500, real_size = 32946016 2024/11/24-00:01:37.900065 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285209.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929114 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.46 avg-zval = 33.29 2024/11/24-00:01:37.900832 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285209.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:01:38.756462 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285209.sst): second pass time = 1.32's, 81.955'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.101'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3331.140'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 290.658'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35130.135'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.955'MB/sec zip pipeline throughput = 81.955'MB/sec entries = 929114 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.46 avg-zval = 33.29 usrkeys = 929114 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.46 avg-zval = 33.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929114 bytes = 3135776 } seq expand size = 6503798 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4449 value = 3.4978 dict = 1.00 all = 3.6339 } Zip/UnZip{ index = 0.0957 value = 0.2859 dict = 1.00 all = 0.2752 } ---------------------------- total value len = 9.422373 GB avg = 0.165 KB (by entry num) total key len = 1.570603 GB avg = 0.028 KB total ukey len = 1.090579 GB avg = 0.016 KB total ukey num = 0.070185486 Billion total entry num = 0.056993215 Billion write speed all = 0.202126758 MB/sec (with seq num) write speed all = 0.193743329 MB/sec (without seq num) 2024/11/24-00:01:38.808841 139957634799168 [compaction_job.cc:1948] [default] [JOB 38245] Generated table #285209: 929114 keys, 36934580 bytes, temperature: kWarm 2024/11/24-00:01:38.808914 139957634799168 EVENT_LOG_v1 {"time_micros": 1732377698808878, "cf_name": "default", "job": 38245, "event": "table_file_creation", "file_number": 285209, "file_size": 36934580, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178559, "largest_seqno": 4337854980, "table_properties": {"data_size": 30934480, "index_size": 1779072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26015192, "raw_average_key_size": 28, "raw_value_size": 108202608, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929114, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377695, "oldest_key_time": 0, "file_creation_time": 1732377695, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285209, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:01:38.855601 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285210.sst) /dev/shm/ToplingTemp/Topling-222267-M5n8tX 2024/11/24-00:01:39.882295 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285210.sst): first pass time = 1.03's, 130.734'MB/sec 2024/11/24-00:01:41.193932 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285210.sst): old prealloc_size = 147639500, real_size = 33322848 2024/11/24-00:01:41.194498 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285210.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926686 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.84 avg-zval = 33.79 2024/11/24-00:01:41.195267 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285210.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:01:42.034526 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285210.sst): second pass time = 1.31's, 82.594'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.380'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3276.951'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 297.785'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34317.946'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.594'MB/sec zip pipeline throughput = 82.594'MB/sec entries = 926686 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.84 avg-zval = 33.79 usrkeys = 926686 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.84 avg-zval = 33.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926686 bytes = 3127584 } seq expand size = 6486802 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4451 value = 3.4573 dict = 1.00 all = 3.5979 } Zip/UnZip{ index = 0.0957 value = 0.2892 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 9.537130 GB avg = 0.165 KB (by entry num) total key len = 1.596550 GB avg = 0.028 KB total ukey len = 1.109113 GB avg = 0.016 KB total ukey num = 0.071112172 Billion total entry num = 0.057919901 Billion write speed all = 0.204701540 MB/sec (with seq num) write speed all = 0.196182314 MB/sec (without seq num) 2024/11/24-00:01:42.085140 139957634799168 [compaction_job.cc:1948] [default] [JOB 38245] Generated table #285210: 926686 keys, 37304818 bytes, temperature: kWarm 2024/11/24-00:01:42.085220 139957634799168 EVENT_LOG_v1 {"time_micros": 1732377702085183, "cf_name": "default", "job": 38245, "event": "table_file_creation", "file_number": 285210, "file_size": 37304818, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178575, "largest_seqno": 4337855106, "table_properties": {"data_size": 31316624, "index_size": 1774400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947208, "raw_average_key_size": 28, "raw_value_size": 108270520, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926686, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377698, "oldest_key_time": 0, "file_creation_time": 1732377698, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285210, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:01:42.126270 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285211.sst) /dev/shm/ToplingTemp/Topling-222267-ioVfvZ 2024/11/24-00:01:43.103992 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285211.sst): first pass time = 0.98's, 137.282'MB/sec 2024/11/24-00:01:44.429347 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285211.sst): old prealloc_size = 147639500, real_size = 33509280 2024/11/24-00:01:44.429903 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285211.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925088 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.09 avg-zval = 34.07 2024/11/24-00:01:44.430668 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285211.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:01:45.284923 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285211.sst): second pass time = 1.32's, 81.794'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.816'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3308.534'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 297.305'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32820.886'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.794'MB/sec zip pipeline throughput = 81.794'MB/sec entries = 925088 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.09 avg-zval = 34.07 usrkeys = 925088 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.09 avg-zval = 34.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925088 bytes = 3122192 } seq expand size = 6475616 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5120 value = 3.4366 dict = 1.00 all = 3.5804 } Zip/UnZip{ index = 0.0951 value = 0.2910 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 9.651921 GB avg = 0.164 KB (by entry num) total key len = 1.622452 GB avg = 0.028 KB total ukey len = 1.127615 GB avg = 0.016 KB total ukey num = 0.072037260 Billion total entry num = 0.058844989 Billion write speed all = 0.207275914 MB/sec (with seq num) write speed all = 0.198621137 MB/sec (without seq num) 2024/11/24-00:01:45.337829 139957634799168 [compaction_job.cc:1948] [default] [JOB 38245] Generated table #285211: 925088 keys, 37487152 bytes, temperature: kWarm 2024/11/24-00:01:45.337898 139957634799168 EVENT_LOG_v1 {"time_micros": 1732377705337862, "cf_name": "default", "job": 38245, "event": "table_file_creation", "file_number": 285211, "file_size": 37487152, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4337855016, "table_properties": {"data_size": 31517776, "index_size": 1760064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25902464, "raw_average_key_size": 28, "raw_value_size": 108315392, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925088, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377702, "oldest_key_time": 0, "file_creation_time": 1732377702, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285211, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:01:45.375153 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285212.sst) /dev/shm/ToplingTemp/Topling-222267-o2hiB0 2024/11/24-00:01:45.958450 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285212.sst): first pass time = 0.58's, 52.753'MB/sec 2024/11/24-00:01:46.253867 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285212.sst): old prealloc_size = 147639500, real_size = 8018080 2024/11/24-00:01:46.254101 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285212.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 216533 prefix = 4 raw-key = 0.0035 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0247 GB zip-val = 0.0075 GB avg-val = 114.09 avg-zval = 34.86 2024/11/24-00:01:46.254330 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285212.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.031 GB 2024/11/24-00:01:46.445898 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285212.sst): second pass time = 0.29's, 83.842'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 46.341'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1975.087'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.335'MB/sec write SST data time = 0.03's, 293.043'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.255'MB, throughput = 8285.492'MB/sec zip my value time = 0.29's, unzip length = 0.025'GB zip my value throughput = 83.842'MB/sec zip pipeline throughput = 83.842'MB/sec entries = 216533 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.09 avg-zval = 34.86 usrkeys = 216533 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.09 avg-zval = 34.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 216533 bytes = 676688 } seq expand size = 1515731 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0043 GB value = 0.0247 GB dict = 0.26 MB all = 0.0308 GB } __ZipSize{ index = 0.0004 GB value = 0.0075 GB dict = 0.26 MB all = 0.0089 GB } UnZip/Zip{ index = 10.4167 value = 3.2730 dict = 1.00 all = 3.4579 } Zip/UnZip{ index = 0.0960 value = 0.3055 dict = 1.00 all = 0.2892 } ---------------------------- total value len = 9.678142 GB avg = 0.164 KB (by entry num) total key len = 1.628515 GB avg = 0.028 KB total ukey len = 1.131945 GB avg = 0.016 KB total ukey num = 0.072253793 Billion total entry num = 0.059061522 Billion write speed all = 0.207865002 MB/sec (with seq num) write speed all = 0.199178563 MB/sec (without seq num) 2024/11/24-00:01:46.463208 139957634799168 [compaction_job.cc:1948] [default] [JOB 38245] Generated table #285212: 216533 keys, 8897961 bytes, temperature: kWarm 2024/11/24-00:01:46.463287 139957634799168 EVENT_LOG_v1 {"time_micros": 1732377706463243, "cf_name": "default", "job": 38245, "event": "table_file_creation", "file_number": 285212, "file_size": 8897961, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4306404542, "largest_seqno": 4337855031, "table_properties": {"data_size": 7548048, "index_size": 415744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6062924, "raw_average_key_size": 28, "raw_value_size": 24704934, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 216533, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377705, "oldest_key_time": 0, "file_creation_time": 1732377705, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285212, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:01:46.489565 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285213.sst) /dev/shm/ToplingTemp/Topling-222267-QsvEDY 2024/11/24-00:01:47.037139 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285213.sst): first pass time = 0.55's, 245.134'MB/sec 2024/11/24-00:01:48.393195 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285213.sst): old prealloc_size = 147639500, real_size = 33055808 2024/11/24-00:01:48.393768 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285213.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925199 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 117.07 avg-zval = 33.58 2024/11/24-00:01:48.394539 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285213.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:01:49.238223 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285213.sst): second pass time = 1.36's, 79.923'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.826'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3210.537'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 289.815'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31926.856'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.923'MB/sec zip pipeline throughput = 79.923'MB/sec entries = 925199 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.07 avg-zval = 33.58 usrkeys = 925199 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.07 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925199 bytes = 3122560 } seq expand size = 6476393 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5140 value = 3.4867 dict = 1.00 all = 3.6242 } Zip/UnZip{ index = 0.0951 value = 0.2868 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 9.792931 GB avg = 0.163 KB (by entry num) total key len = 1.654421 GB avg = 0.028 KB total ukey len = 1.150449 GB avg = 0.016 KB total ukey num = 0.073178992 Billion total entry num = 0.059986721 Billion write speed all = 0.210440764 MB/sec (with seq num) write speed all = 0.201618705 MB/sec (without seq num) 2024/11/24-00:01:49.290654 139957634799168 [compaction_job.cc:1948] [default] [JOB 38245] Generated table #285213: 925199 keys, 37033668 bytes, temperature: kWarm 2024/11/24-00:01:49.290724 139957634799168 EVENT_LOG_v1 {"time_micros": 1732377709290689, "cf_name": "default", "job": 38245, "event": "table_file_creation", "file_number": 285213, "file_size": 37033668, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4337855239, "table_properties": {"data_size": 31064368, "index_size": 1759936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25905572, "raw_average_key_size": 28, "raw_value_size": 108312274, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925199, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377706, "oldest_key_time": 0, "file_creation_time": 1732377706, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285213, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:01:49.329289 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285214.sst) /dev/shm/ToplingTemp/Topling-222267-nOYnQ0 2024/11/24-00:01:50.357175 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285214.sst): first pass time = 1.03's, 130.583'MB/sec 2024/11/24-00:01:51.705809 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285214.sst): old prealloc_size = 147639500, real_size = 33380192 2024/11/24-00:01:51.706378 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285214.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926754 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.83 avg-zval = 33.87 2024/11/24-00:01:51.707144 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285214.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:01:52.562517 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285214.sst): second pass time = 1.35's, 80.329'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.613'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3251.941'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 295.238'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34677.472'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.329'MB/sec zip pipeline throughput = 80.329'MB/sec entries = 926754 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.87 usrkeys = 926754 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926754 bytes = 3127808 } seq expand size = 6487278 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5107 value = 3.4497 dict = 1.00 all = 3.5923 } Zip/UnZip{ index = 0.0951 value = 0.2899 dict = 1.00 all = 0.2784 } ---------------------------- total value len = 9.907687 GB avg = 0.163 KB (by entry num) total key len = 1.680370 GB avg = 0.028 KB total ukey len = 1.168984 GB avg = 0.016 KB total ukey num = 0.074105746 Billion total entry num = 0.060913475 Billion write speed all = 0.213014379 MB/sec (with seq num) write speed all = 0.204056572 MB/sec (without seq num) 2024/11/24-00:01:52.615018 139957634799168 [compaction_job.cc:1948] [default] [JOB 38245] Generated table #285214: 926754 keys, 37362574 bytes, temperature: kWarm 2024/11/24-00:01:52.615092 139957634799168 EVENT_LOG_v1 {"time_micros": 1732377712615055, "cf_name": "default", "job": 38245, "event": "table_file_creation", "file_number": 285214, "file_size": 37362574, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4337855264, "table_properties": {"data_size": 31384848, "index_size": 1763456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25949112, "raw_average_key_size": 28, "raw_value_size": 108268712, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926754, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377709, "oldest_key_time": 0, "file_creation_time": 1732377709, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285214, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:01:52.656457 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285215.sst) /dev/shm/ToplingTemp/Topling-222267-uBQtY0 2024/11/24-00:01:52.672333 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285215.sst): first pass time = 0.02's, 184.105'MB/sec 2024/11/24-00:01:52.711616 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285215.sst): old prealloc_size = 147639500, real_size = 917888 2024/11/24-00:01:52.711680 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285215.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 20543 prefix = 4 raw-key = 0.0003 GB zip-key = 0.0000 GB avg-key = 16.00 avg-zkey = 1.99 raw-val = 0.0023 GB zip-val = 0.0009 GB avg-val = 114.00 avg-zval = 42.43 2024/11/24-00:01:52.711732 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285215.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0000 GB, waited 0.000 sec, Key+Value = 0.003 GB 2024/11/24-00:01:52.728212 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00/285215.sst): second pass time = 0.04's, 60.400'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.02's, 59.977'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 942.154'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.389'MB/sec write SST data time = 0.00's, 297.538'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.023'MB, throughput = 994.495'MB/sec zip my value time = 0.04's, unzip length = 0.002'GB zip my value throughput = 60.400'MB/sec zip pipeline throughput = 60.400'MB/sec entries = 20543 avg-key = 28.00 avg-zkey = 1.99 avg-val = 114.00 avg-zval = 42.43 usrkeys = 20543 avg-key = 20.00 avg-zkey = 1.99 avg-val = 121.00 avg-zval = 42.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 20543 bytes = 56512 } seq expand size = 143801 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0004 GB value = 0.0023 GB dict = 0.02 MB all = 0.0029 GB } __ZipSize{ index = 0.0000 GB value = 0.0009 GB dict = 0.02 MB all = 0.0010 GB } UnZip/Zip{ index = 10.0622 value = 2.6865 dict = 1.00 all = 2.9347 } Zip/UnZip{ index = 0.0994 value = 0.3722 dict = 1.00 all = 0.3408 } ---------------------------- total value len = 9.910172 GB avg = 0.163 KB (by entry num) total key len = 1.680945 GB avg = 0.028 KB total ukey len = 1.169395 GB avg = 0.016 KB total ukey num = 0.074126289 Billion total entry num = 0.060934018 Billion write speed all = 0.213069996 MB/sec (with seq num) write speed all = 0.204109196 MB/sec (without seq num) 2024/11/24-00:01:52.742120 139957634799168 [compaction_job.cc:1948] [default] [JOB 38245] Generated table #285215: 20543 keys, 994005 bytes, temperature: kWarm 2024/11/24-00:01:52.742195 139957634799168 EVENT_LOG_v1 {"time_micros": 1732377712742151, "cf_name": "default", "job": 38245, "event": "table_file_creation", "file_number": 285215, "file_size": 994005, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4315361563, "largest_seqno": 4318518445, "table_properties": {"data_size": 871728, "index_size": 40832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 575204, "raw_average_key_size": 28, "raw_value_size": 2341902, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 20543, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732377712, "oldest_key_time": 0, "file_creation_time": 1732377712, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285215, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:01:52.750902 139957634799168 [compaction_job.cc:703] job-38245: subcompact[0], size: 0.344931 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38245/att-00 [285205,285206,285207,285208,285209,285210,285211,285212,285213,285214,285215] 2024/11/24-00:01:52.811754 139957634799168 (Original Log Time 2024/11/24-00:01:22.111496) EVENT_LOG_v1 {"time_micros": 1732377682111456, "job": 38245, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [285157], "files_L3": [285115, 285116, 285117, 285118, 285119, 285120, 285102, 285103, 285104, 285105, 285191], "score": 0.503782, "input_data_size": 368252079, "oldest_snapshot_seqno": 4338157768} 2024/11/24-00:01:52.811758 139957634799168 (Original Log Time 2024/11/24-00:01:52.810787) [compaction_job.cc:2022] [default] [JOB 38245] Compacted 1@2 + 11@3 files to L3 => 344930938 bytes 2024/11/24-00:01:52.811759 139957634799168 (Original Log Time 2024/11/24-00:01:52.811689) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 11 0 0 0] max score 0.01, MB/sec: 12.0 rd, 11.3 wr, level 3, files in(1, 11) out(11 +0 blob) MB in(27.2, 324.0 +0.0 blob) out(329.0 +0.0 blob), read-write-amplify(25.0) write-amplify(12.1) OK, records in: 8690656, records dropped: 114802 output_compression: Snappy 2024/11/24-00:01:52.811765 139957634799168 (Original Log Time 2024/11/24-00:01:52.811734) EVENT_LOG_v1 {"time_micros": 1732377712811713, "cf": "default", "job": 38245, "event": "compaction_finished", "compaction_time_micros": 30639414, "compaction_time_cpu_micros": 13006372, "output_level": 3, "num_output_files": 11, "total_output_size": 344930938, "num_input_records": 8690656, "num_output_records": 8575854, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 239085877, "file_range_sync_nanos": 0, "file_fsync_nanos": 7273, "file_prepare_write_nanos": 29915, "lsm_state": [0, 0, 0, 11, 0, 0, 0]} 2024/11/24-00:01:52.933706 139957634799168 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732377681,"executesMs":30839,"compactionJobId":38245,"attempt":0,"compactionInputRawBytes":1257947252,"compactionInputZipBytes":368252079,"compactionOutputRawBytes":0,"compactionOutputZipBytes":344930938,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-9d2ac14802b86e5a39fd3ced2274773b-46ac631453089f04-00"}}, response = 2024/11/24-00:01:52.933795 139957634799168 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38245/att-00: olev 3, work 30.839 s, result 6.144 ms, install 0.996 ms, input{raw 1.172 GiB zip 351.193 MiB}