2024/11/24-01:51:10.462435 139957282305600 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00: fork to child time = 0.007795 sec 2024/11/24-01:51:10.463131 139957282305600 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00 2024/11/24-01:51:10.470506 139957282305600 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00 2024/11/24-01:51:10.471434 139957282305600 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-01:51:10.471644 139957282305600 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-01:51:10.525411 139957282305600 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39640/att-00/MANIFEST-206347 2024/11/24-01:51:10.552613 139957282305600 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39640/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 287856, last_sequence is 4381418442, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 287688 2024/11/24-01:51:10.552620 139957282305600 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-01:51:10.552707 139957282305600 [version_set.cc:5721] Creating manifest 287856 2024/11/24-01:51:10.586008 139957282305600 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-01:51:10.607097 139957282305600 [compaction_job.cc:2437] [default] [JOB 39640] Compacting 1@2 + 14@3 files to L3, score 0.76, subcompactions 1 : 1 2024/11/24-01:51:10.607111 139957282305600 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 29701 Base level 2, inputs: [287824(27MB)], [287743(33MB) 287744(33MB) 287745(34MB) 287746(34MB) 287747(34MB) 287748(34MB) 287749(35MB) 287750(35MB) 287751(35MB) 287752(35MB) 287753(11MB) 287754(33MB) 287755(33MB) 287847(33MB)] 2024/11/24-01:51:10.617367 139957282305600 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:51:10.617401 139957282305600 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:51:10.617417 139957282305600 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:51:10.617422 139957282305600 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:51:10.617434 139957282305600 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:51:11.182971 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287857.sst) /dev/shm/ToplingTemp/Topling-241782-lROk2A 2024/11/24-01:51:11.748424 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287857.sst): first pass time = 0.57's, 237.390'MB/sec 2024/11/24-01:51:13.293720 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287857.sst): old prealloc_size = 147639500, real_size = 31052448 2024/11/24-01:51:13.294312 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287857.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924669 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0291 GB avg-val = 117.15 avg-zval = 31.42 2024/11/24-01:51:13.295106 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287857.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:14.113451 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287857.sst): second pass time = 1.54's, 70.145'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.731'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3107.274'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 267.841'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31650.433'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.145'MB/sec zip pipeline throughput = 116.299'MB/sec entries = 924669 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.15 avg-zval = 31.42 usrkeys = 924669 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.15 avg-zval = 31.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924669 bytes = 3120768 } seq expand size = 6472683 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4446 value = 3.7289 dict = 1.00 all = 3.8318 } Zip/UnZip{ index = 0.0957 value = 0.2682 dict = 1.00 all = 0.2610 } ---------------------------- total value len = 8.963419 GB avg = 0.168 KB (by entry num) total key len = 1.466712 GB avg = 0.028 KB total ukey len = 1.016371 GB avg = 0.015 KB total ukey num = 0.066475104 Billion total entry num = 0.053282833 Billion write speed all = 0.171092625 MB/sec (with seq num) write speed all = 0.164100345 MB/sec (without seq num) 2024/11/24-01:51:14.157509 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287857: 924669 keys, 35027432 bytes, temperature: kWarm 2024/11/24-01:51:14.157606 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384274157547, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287857, "file_size": 35027432, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4259662629, "largest_seqno": 4381223196, "table_properties": {"data_size": 29050448, "index_size": 1770624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25890732, "raw_average_key_size": 28, "raw_value_size": 108326998, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924669, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384271, "oldest_key_time": 0, "file_creation_time": 1732384271, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287857, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:14.194872 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287858.sst) /dev/shm/ToplingTemp/Topling-241782-k1n9YB 2024/11/24-01:51:15.073749 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287858.sst): first pass time = 0.88's, 152.724'MB/sec 2024/11/24-01:51:16.538835 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287858.sst): old prealloc_size = 147639500, real_size = 31673792 2024/11/24-01:51:16.539405 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287858.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924663 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 117.15 avg-zval = 32.10 2024/11/24-01:51:16.540160 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287858.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:17.361627 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287858.sst): second pass time = 1.46's, 73.978'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.330'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3203.421'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 274.030'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35038.071'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 73.978'MB/sec zip pipeline throughput = 78.615'MB/sec entries = 924663 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 32.10 usrkeys = 924663 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 32.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924663 bytes = 3120752 } seq expand size = 6472641 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5129 value = 3.6494 dict = 1.00 all = 3.7648 } Zip/UnZip{ index = 0.0951 value = 0.2740 dict = 1.00 all = 0.2656 } ---------------------------- total value len = 9.078219 GB avg = 0.167 KB (by entry num) total key len = 1.492602 GB avg = 0.028 KB total ukey len = 1.034865 GB avg = 0.015 KB total ukey num = 0.067399767 Billion total entry num = 0.054207496 Billion write speed all = 0.173391228 MB/sec (with seq num) write speed all = 0.166277984 MB/sec (without seq num) 2024/11/24-01:51:17.405382 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287858: 924663 keys, 35650548 bytes, temperature: kWarm 2024/11/24-01:51:17.405453 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384277405416, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287858, "file_size": 35650548, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4259662645, "largest_seqno": 4381223975, "table_properties": {"data_size": 29683312, "index_size": 1759104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25890564, "raw_average_key_size": 28, "raw_value_size": 108327230, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924663, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384274, "oldest_key_time": 0, "file_creation_time": 1732384274, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287858, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:17.443685 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287859.sst) /dev/shm/ToplingTemp/Topling-241782-3mf0rB 2024/11/24-01:51:18.425987 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287859.sst): first pass time = 0.98's, 136.642'MB/sec 2024/11/24-01:51:19.948376 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287859.sst): old prealloc_size = 147639500, real_size = 32163232 2024/11/24-01:51:19.948930 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287859.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924988 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0302 GB avg-val = 117.10 avg-zval = 32.62 2024/11/24-01:51:19.949705 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287859.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:20.785555 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287859.sst): second pass time = 1.52's, 71.187'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.171'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3305.101'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 276.788'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35638.948'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.187'MB/sec zip pipeline throughput = 118.340'MB/sec entries = 924988 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 32.62 usrkeys = 924988 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 32.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924988 bytes = 3121856 } seq expand size = 6474916 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5105 value = 3.5901 dict = 1.00 all = 3.7138 } Zip/UnZip{ index = 0.0951 value = 0.2785 dict = 1.00 all = 0.2693 } ---------------------------- total value len = 9.193012 GB avg = 0.167 KB (by entry num) total key len = 1.518502 GB avg = 0.028 KB total ukey len = 1.053364 GB avg = 0.015 KB total ukey num = 0.068324755 Billion total entry num = 0.055132484 Billion write speed all = 0.175689119 MB/sec (with seq num) write speed all = 0.168454902 MB/sec (without seq num) 2024/11/24-01:51:20.829496 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287859: 924988 keys, 36140522 bytes, temperature: kWarm 2024/11/24-01:51:20.829563 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384280829529, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287859, "file_size": 36140522, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178503, "largest_seqno": 4381223218, "table_properties": {"data_size": 30171664, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25899664, "raw_average_key_size": 28, "raw_value_size": 108318184, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924988, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384277, "oldest_key_time": 0, "file_creation_time": 1732384277, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287859, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:20.869622 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287860.sst) /dev/shm/ToplingTemp/Topling-241782-CXDN5A 2024/11/24-01:51:21.735604 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287860.sst): first pass time = 0.87's, 154.996'MB/sec 2024/11/24-01:51:23.260996 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287860.sst): old prealloc_size = 147639500, real_size = 32413296 2024/11/24-01:51:23.261516 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287860.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924918 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 117.11 avg-zval = 32.89 2024/11/24-01:51:23.262274 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287860.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:24.092576 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287860.sst): second pass time = 1.52's, 71.053'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.761'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3563.015'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 284.809'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32168.973'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.053'MB/sec zip pipeline throughput = 108.379'MB/sec entries = 924918 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.11 avg-zval = 32.89 usrkeys = 924918 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.11 avg-zval = 32.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924918 bytes = 3121616 } seq expand size = 6474426 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5139 value = 3.5605 dict = 1.00 all = 3.6883 } Zip/UnZip{ index = 0.0951 value = 0.2809 dict = 1.00 all = 0.2711 } ---------------------------- total value len = 9.307807 GB avg = 0.166 KB (by entry num) total key len = 1.544400 GB avg = 0.028 KB total ukey len = 1.071863 GB avg = 0.015 KB total ukey num = 0.069249673 Billion total entry num = 0.056057402 Billion write speed all = 0.177987083 MB/sec (with seq num) write speed all = 0.170631902 MB/sec (without seq num) 2024/11/24-01:51:24.137936 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287860: 924918 keys, 36390494 bytes, temperature: kWarm 2024/11/24-01:51:24.138024 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384284137970, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287860, "file_size": 36390494, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178518, "largest_seqno": 4381224079, "table_properties": {"data_size": 30422432, "index_size": 1759424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897704, "raw_average_key_size": 28, "raw_value_size": 108320080, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924918, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384280, "oldest_key_time": 0, "file_creation_time": 1732384280, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287860, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:24.176582 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287861.sst) /dev/shm/ToplingTemp/Topling-241782-7vwQGB 2024/11/24-01:51:25.065241 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287861.sst): first pass time = 0.89's, 151.041'MB/sec 2024/11/24-01:51:26.638442 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287861.sst): old prealloc_size = 147639500, real_size = 32780128 2024/11/24-01:51:26.638993 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924942 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.11 avg-zval = 33.29 2024/11/24-01:51:26.639745 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287861.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:27.487424 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287861.sst): second pass time = 1.57's, 68.910'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.298'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3313.982'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 281.454'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34593.579'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.910'MB/sec zip pipeline throughput = 113.931'MB/sec entries = 924942 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.11 avg-zval = 33.29 usrkeys = 924942 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.11 avg-zval = 33.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924942 bytes = 3121696 } seq expand size = 6474594 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5134 value = 3.5181 dict = 1.00 all = 3.6515 } Zip/UnZip{ index = 0.0951 value = 0.2842 dict = 1.00 all = 0.2739 } ---------------------------- total value len = 9.422601 GB avg = 0.165 KB (by entry num) total key len = 1.570298 GB avg = 0.028 KB total ukey len = 1.090362 GB avg = 0.016 KB total ukey num = 0.070174615 Billion total entry num = 0.056982344 Billion write speed all = 0.180284542 MB/sec (with seq num) write speed all = 0.172808417 MB/sec (without seq num) 2024/11/24-01:51:27.537769 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287861: 924942 keys, 36757106 bytes, temperature: kWarm 2024/11/24-01:51:27.537828 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384287537795, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287861, "file_size": 36757106, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178530, "largest_seqno": 4381223240, "table_properties": {"data_size": 30789136, "index_size": 1759552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25898376, "raw_average_key_size": 28, "raw_value_size": 108319444, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924942, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384284, "oldest_key_time": 0, "file_creation_time": 1732384284, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:27.574906 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287862.sst) /dev/shm/ToplingTemp/Topling-241782-fHbPQB 2024/11/24-01:51:28.433561 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287862.sst): first pass time = 0.86's, 156.318'MB/sec 2024/11/24-01:51:30.054192 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287862.sst): old prealloc_size = 147639500, real_size = 33056928 2024/11/24-01:51:30.054754 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925065 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 117.09 avg-zval = 33.57 2024/11/24-01:51:30.055511 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287862.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:30.910383 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287862.sst): second pass time = 1.62's, 66.925'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.256'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3270.362'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 280.220'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32778.098'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.925'MB/sec zip pipeline throughput = 117.039'MB/sec entries = 925065 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.09 avg-zval = 33.57 usrkeys = 925065 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.09 avg-zval = 33.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925065 bytes = 3122112 } seq expand size = 6475455 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4453 value = 3.4880 dict = 1.00 all = 3.6242 } Zip/UnZip{ index = 0.0957 value = 0.2867 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 9.537392 GB avg = 0.165 KB (by entry num) total key len = 1.596200 GB avg = 0.028 KB total ukey len = 1.108863 GB avg = 0.016 KB total ukey num = 0.071099680 Billion total entry num = 0.057907409 Billion write speed all = 0.182581675 MB/sec (with seq num) write speed all = 0.174984608 MB/sec (without seq num) 2024/11/24-01:51:30.993429 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287862: 925065 keys, 37034146 bytes, temperature: kWarm 2024/11/24-01:51:30.993476 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384290993450, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287862, "file_size": 37034146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178548, "largest_seqno": 4381224463, "table_properties": {"data_size": 31054224, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25901820, "raw_average_key_size": 28, "raw_value_size": 108316042, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925065, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384287, "oldest_key_time": 0, "file_creation_time": 1732384287, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:31.039529 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287863.sst) /dev/shm/ToplingTemp/Topling-241782-1DlLLB 2024/11/24-01:51:31.884322 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287863.sst): first pass time = 0.84's, 158.882'MB/sec 2024/11/24-01:51:33.468883 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287863.sst): old prealloc_size = 147639500, real_size = 32982544 2024/11/24-01:51:33.469465 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287863.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925883 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.96 avg-zval = 33.47 2024/11/24-01:51:33.470217 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287863.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:34.325753 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287863.sst): second pass time = 1.58's, 68.377'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.137'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3224.175'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 282.988'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25205.497'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.377'MB/sec zip pipeline throughput = 122.173'MB/sec entries = 925883 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 33.47 usrkeys = 925883 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 33.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925883 bytes = 3124864 } seq expand size = 6481181 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5134 value = 3.4945 dict = 1.00 all = 3.6312 } Zip/UnZip{ index = 0.0951 value = 0.2862 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 9.652167 GB avg = 0.164 KB (by entry num) total key len = 1.622125 GB avg = 0.028 KB total ukey len = 1.127381 GB avg = 0.016 KB total ukey num = 0.072025563 Billion total entry num = 0.058833292 Billion write speed all = 0.184878666 MB/sec (with seq num) write speed all = 0.177160561 MB/sec (without seq num) 2024/11/24-01:51:34.374800 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287863: 925883 keys, 36962166 bytes, temperature: kWarm 2024/11/24-01:51:34.374858 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384294374827, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287863, "file_size": 36962166, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178559, "largest_seqno": 4381223250, "table_properties": {"data_size": 30989568, "index_size": 1761344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924724, "raw_average_key_size": 28, "raw_value_size": 108293050, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925883, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384291, "oldest_key_time": 0, "file_creation_time": 1732384291, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287863, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:34.409300 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287864.sst) /dev/shm/ToplingTemp/Topling-241782-AROdED 2024/11/24-01:51:35.246292 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287864.sst): first pass time = 0.84's, 160.364'MB/sec 2024/11/24-01:51:36.842359 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287864.sst): old prealloc_size = 147639500, real_size = 33302368 2024/11/24-01:51:36.842912 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287864.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925968 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.95 avg-zval = 33.81 2024/11/24-01:51:36.843662 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287864.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:37.697657 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287864.sst): second pass time = 1.60's, 67.881'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.591'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3302.682'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 287.831'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28502.968'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.881'MB/sec zip pipeline throughput = 125.833'MB/sec entries = 925968 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.95 avg-zval = 33.81 usrkeys = 925968 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.95 avg-zval = 33.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925968 bytes = 3125152 } seq expand size = 6481776 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5136 value = 3.4587 dict = 1.00 all = 3.6001 } Zip/UnZip{ index = 0.0951 value = 0.2891 dict = 1.00 all = 0.2778 } ---------------------------- total value len = 9.766939 GB avg = 0.163 KB (by entry num) total key len = 1.648052 GB avg = 0.028 KB total ukey len = 1.145900 GB avg = 0.016 KB total ukey num = 0.072951531 Billion total entry num = 0.059759260 Billion write speed all = 0.187175543 MB/sec (with seq num) write speed all = 0.179336397 MB/sec (without seq num) 2024/11/24-01:51:37.747993 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287864: 925968 keys, 37282156 bytes, temperature: kWarm 2024/11/24-01:51:37.748063 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384297748028, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287864, "file_size": 37282156, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178575, "largest_seqno": 4381223267, "table_properties": {"data_size": 31309200, "index_size": 1761472, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927104, "raw_average_key_size": 28, "raw_value_size": 108290628, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925968, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384294, "oldest_key_time": 0, "file_creation_time": 1732384294, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287864, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:37.784013 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287865.sst) /dev/shm/ToplingTemp/Topling-241782-4WEpeA 2024/11/24-01:51:38.670212 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287865.sst): first pass time = 0.89's, 151.460'MB/sec 2024/11/24-01:51:40.254516 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287865.sst): old prealloc_size = 147639500, real_size = 33296880 2024/11/24-01:51:40.255053 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287865.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925910 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.96 avg-zval = 33.81 2024/11/24-01:51:40.256709 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287865.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:41.117177 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287865.sst): second pass time = 1.58's, 68.433'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.212'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3416.882'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 285.039'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25985.664'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.433'MB/sec zip pipeline throughput = 117.492'MB/sec entries = 925910 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 33.81 usrkeys = 925910 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 33.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925910 bytes = 3124960 } seq expand size = 6481370 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5125 value = 3.4594 dict = 1.00 all = 3.6006 } Zip/UnZip{ index = 0.0951 value = 0.2891 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 9.881713 GB avg = 0.163 KB (by entry num) total key len = 1.673977 GB avg = 0.028 KB total ukey len = 1.164418 GB avg = 0.016 KB total ukey num = 0.073877441 Billion total entry num = 0.060685170 Billion write speed all = 0.189472012 MB/sec (with seq num) write speed all = 0.181511853 MB/sec (without seq num) 2024/11/24-01:51:41.163566 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287865: 925910 keys, 37276594 bytes, temperature: kWarm 2024/11/24-01:51:41.163631 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384301163599, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287865, "file_size": 37276594, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178589, "largest_seqno": 4381223861, "table_properties": {"data_size": 31303712, "index_size": 1761536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25925480, "raw_average_key_size": 28, "raw_value_size": 108292360, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925910, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384297, "oldest_key_time": 0, "file_creation_time": 1732384297, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287865, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:41.202952 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287866.sst) /dev/shm/ToplingTemp/Topling-241782-Z7Z6iA 2024/11/24-01:51:42.080661 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287866.sst): first pass time = 0.88's, 152.928'MB/sec 2024/11/24-01:51:43.630419 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287866.sst): old prealloc_size = 147639500, real_size = 33515152 2024/11/24-01:51:43.630952 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287866.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925980 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 116.95 avg-zval = 34.03 2024/11/24-01:51:43.631702 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287866.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:44.498563 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287866.sst): second pass time = 1.55's, 69.928'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.191'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3421.572'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 288.406'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33374.143'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.928'MB/sec zip pipeline throughput = 104.727'MB/sec entries = 925980 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.95 avg-zval = 34.03 usrkeys = 925980 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.95 avg-zval = 34.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925980 bytes = 3125200 } seq expand size = 6481860 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4446 value = 3.4367 dict = 1.00 all = 3.5796 } Zip/UnZip{ index = 0.0957 value = 0.2910 dict = 1.00 all = 0.2794 } ---------------------------- total value len = 9.996485 GB avg = 0.162 KB (by entry num) total key len = 1.699905 GB avg = 0.028 KB total ukey len = 1.182938 GB avg = 0.016 KB total ukey num = 0.074803421 Billion total entry num = 0.061611150 Billion write speed all = 0.191768353 MB/sec (with seq num) write speed all = 0.183687180 MB/sec (without seq num) 2024/11/24-01:51:44.545262 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287866: 925980 keys, 37495260 bytes, temperature: kWarm 2024/11/24-01:51:44.545331 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384304545297, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287866, "file_size": 37495260, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4381223299, "table_properties": {"data_size": 31510336, "index_size": 1773120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927440, "raw_average_key_size": 28, "raw_value_size": 108290408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925980, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384301, "oldest_key_time": 0, "file_creation_time": 1732384301, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287866, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:44.580077 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287867.sst) /dev/shm/ToplingTemp/Topling-241782-8nntgA 2024/11/24-01:51:45.346298 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287867.sst): first pass time = 0.77's, 73.968'MB/sec 2024/11/24-01:51:45.909480 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287867.sst): old prealloc_size = 147639500, real_size = 14230336 2024/11/24-01:51:45.909753 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287867.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 394931 prefix = 4 raw-key = 0.0063 GB zip-key = 0.0008 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0456 GB zip-val = 0.0134 GB avg-val = 115.50 avg-zval = 33.86 2024/11/24-01:51:45.910129 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287867.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.057 GB 2024/11/24-01:51:46.383669 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287867.sst): second pass time = 0.56's, 81.132'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.47's, 33.493'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3061.371'MB/sec (index lex order gen) rebuild zvType time = 0.42's, 0.236'MB/sec write SST data time = 0.06's, 288.411'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.464'MB, throughput = 15437.691'MB/sec zip my value time = 0.56's, unzip length = 0.046'GB zip my value throughput = 81.132'MB/sec zip pipeline throughput = 81.132'MB/sec entries = 394931 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.50 avg-zval = 33.86 usrkeys = 394931 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.50 avg-zval = 33.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 394931 bytes = 1283536 } seq expand size = 2764517 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0079 GB value = 0.0456 GB dict = 0.46 MB all = 0.0567 GB } __ZipSize{ index = 0.0008 GB value = 0.0134 GB dict = 0.46 MB all = 0.0159 GB } UnZip/Zip{ index = 10.4325 value = 3.4106 dict = 1.00 all = 3.5685 } Zip/UnZip{ index = 0.0959 value = 0.2932 dict = 1.00 all = 0.2802 } ---------------------------- total value len = 10.044865 GB avg = 0.162 KB (by entry num) total key len = 1.710963 GB avg = 0.028 KB total ukey len = 1.190836 GB avg = 0.016 KB total ukey num = 0.075198352 Billion total entry num = 0.062006081 Billion write speed all = 0.192736907 MB/sec (with seq num) write speed all = 0.184604185 MB/sec (without seq num) 2024/11/24-01:51:46.409093 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287867: 394931 keys, 15881471 bytes, temperature: kWarm 2024/11/24-01:51:46.409156 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384306409124, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287867, "file_size": 15881471, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4345110091, "largest_seqno": 4381223312, "table_properties": {"data_size": 13374320, "index_size": 757120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 11058068, "raw_average_key_size": 28, "raw_value_size": 45615094, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 394931, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384304, "oldest_key_time": 0, "file_creation_time": 1732384304, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287867, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:46.434646 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287868.sst) /dev/shm/ToplingTemp/Topling-241782-CyvVAC 2024/11/24-01:51:46.968996 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287868.sst): first pass time = 0.53's, 251.196'MB/sec 2024/11/24-01:51:48.323702 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287868.sst): old prealloc_size = 147639500, real_size = 30906960 2024/11/24-01:51:48.324263 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287868.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925002 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.10 avg-zval = 31.25 2024/11/24-01:51:48.325025 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287868.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:49.144102 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287868.sst): second pass time = 1.35's, 79.999'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.520'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3251.961'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 264.143'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36745.114'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.999'MB/sec zip pipeline throughput = 79.999'MB/sec entries = 925002 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.10 avg-zval = 31.25 usrkeys = 925002 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.10 avg-zval = 31.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925002 bytes = 3121904 } seq expand size = 6475014 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4461 value = 3.7474 dict = 1.00 all = 3.8477 } Zip/UnZip{ index = 0.0957 value = 0.2668 dict = 1.00 all = 0.2599 } ---------------------------- total value len = 10.159658 GB avg = 0.161 KB (by entry num) total key len = 1.736863 GB avg = 0.028 KB total ukey len = 1.209336 GB avg = 0.016 KB total ukey num = 0.076123354 Billion total entry num = 0.062931083 Billion write speed all = 0.195034741 MB/sec (with seq num) write speed all = 0.186781068 MB/sec (without seq num) 2024/11/24-01:51:49.188008 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287868: 925002 keys, 34882630 bytes, temperature: kWarm 2024/11/24-01:51:49.188079 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384309188043, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287868, "file_size": 34882630, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4381222935, "table_properties": {"data_size": 28904512, "index_size": 1771008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900056, "raw_average_key_size": 28, "raw_value_size": 108317764, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925002, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384306, "oldest_key_time": 0, "file_creation_time": 1732384306, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287868, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:49.225631 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287869.sst) /dev/shm/ToplingTemp/Topling-241782-t99sWD 2024/11/24-01:51:50.087665 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287869.sst): first pass time = 0.86's, 155.706'MB/sec 2024/11/24-01:51:51.440176 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287869.sst): old prealloc_size = 147639500, real_size = 31369088 2024/11/24-01:51:51.440766 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287869.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925034 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0294 GB avg-val = 117.10 avg-zval = 31.76 2024/11/24-01:51:51.442442 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287869.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:52.257645 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287869.sst): second pass time = 1.35's, 80.144'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.240'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3110.008'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 272.590'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29906.820'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.144'MB/sec zip pipeline throughput = 80.144'MB/sec entries = 925034 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 31.76 usrkeys = 925034 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 31.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925034 bytes = 3122000 } seq expand size = 6475238 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.5117 value = 3.6871 dict = 1.00 all = 3.7972 } Zip/UnZip{ index = 0.0951 value = 0.2712 dict = 1.00 all = 0.2634 } ---------------------------- total value len = 10.274450 GB avg = 0.161 KB (by entry num) total key len = 1.762764 GB avg = 0.028 KB total ukey len = 1.227837 GB avg = 0.016 KB total ukey num = 0.077048388 Billion total entry num = 0.063856117 Billion write speed all = 0.197331228 MB/sec (with seq num) write speed all = 0.188956661 MB/sec (without seq num) 2024/11/24-01:51:52.301827 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287869: 925034 keys, 35346662 bytes, temperature: kWarm 2024/11/24-01:51:52.301901 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384312301860, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287869, "file_size": 35346662, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178655, "largest_seqno": 4381224581, "table_properties": {"data_size": 29377648, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900952, "raw_average_key_size": 28, "raw_value_size": 108316872, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925034, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384309, "oldest_key_time": 0, "file_creation_time": 1732384309, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287869, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:52.338070 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287870.sst) /dev/shm/ToplingTemp/Topling-241782-xptcHA 2024/11/24-01:51:53.416114 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287870.sst): first pass time = 1.08's, 124.505'MB/sec 2024/11/24-01:51:54.767167 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287870.sst): old prealloc_size = 147639500, real_size = 31910528 2024/11/24-01:51:54.767727 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287870.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925057 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0299 GB avg-val = 117.09 avg-zval = 32.34 2024/11/24-01:51:54.768492 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287870.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:51:55.603192 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287870.sst): second pass time = 1.35's, 80.230'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.928'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3269.398'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 274.580'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31908.900'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.230'MB/sec zip pipeline throughput = 80.230'MB/sec entries = 925057 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.09 avg-zval = 32.34 usrkeys = 925057 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.09 avg-zval = 32.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925057 bytes = 3122080 } seq expand size = 6475399 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5116 value = 3.6203 dict = 1.00 all = 3.7399 } Zip/UnZip{ index = 0.0951 value = 0.2762 dict = 1.00 all = 0.2674 } ---------------------------- total value len = 10.389241 GB avg = 0.160 KB (by entry num) total key len = 1.788665 GB avg = 0.028 KB total ukey len = 1.246338 GB avg = 0.016 KB total ukey num = 0.077973445 Billion total entry num = 0.064781174 Billion write speed all = 0.199626722 MB/sec (with seq num) write speed all = 0.191131303 MB/sec (without seq num) 2024/11/24-01:51:55.647477 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287870: 925057 keys, 35888098 bytes, temperature: kWarm 2024/11/24-01:51:55.647537 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384315647504, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287870, "file_size": 35888098, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4381224204, "table_properties": {"data_size": 29919024, "index_size": 1760064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25901596, "raw_average_key_size": 28, "raw_value_size": 108316138, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925057, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384312, "oldest_key_time": 0, "file_creation_time": 1732384312, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287870, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:55.682036 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287871.sst) /dev/shm/ToplingTemp/Topling-241782-c8771C 2024/11/24-01:51:55.697922 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287871.sst): first pass time = 0.02's, 185.845'MB/sec 2024/11/24-01:51:55.736518 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287871.sst): old prealloc_size = 147639500, real_size = 923664 2024/11/24-01:51:55.736571 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287871.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 20750 prefix = 4 raw-key = 0.0003 GB zip-key = 0.0000 GB avg-key = 16.00 avg-zkey = 1.99 raw-val = 0.0024 GB zip-val = 0.0009 GB avg-val = 114.00 avg-zval = 42.26 2024/11/24-01:51:55.736616 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287871.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0000 GB, waited 0.000 sec, Key+Value = 0.003 GB 2024/11/24-01:51:55.753288 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00/287871.sst): second pass time = 0.04's, 62.049'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.02's, 59.692'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1087.804'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.386'MB/sec write SST data time = 0.00's, 305.859'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.024'MB, throughput = 940.206'MB/sec zip my value time = 0.04's, unzip length = 0.002'GB zip my value throughput = 62.049'MB/sec zip pipeline throughput = 62.049'MB/sec entries = 20750 avg-key = 28.00 avg-zkey = 1.99 avg-val = 114.00 avg-zval = 42.26 usrkeys = 20750 avg-key = 20.00 avg-zkey = 1.99 avg-val = 121.00 avg-zval = 42.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 20750 bytes = 57072 } seq expand size = 145250 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0004 GB value = 0.0024 GB dict = 0.02 MB all = 0.0029 GB } __ZipSize{ index = 0.0000 GB value = 0.0009 GB dict = 0.02 MB all = 0.0010 GB } UnZip/Zip{ index = 10.0533 value = 2.6973 dict = 1.00 all = 2.9452 } Zip/UnZip{ index = 0.0995 value = 0.3707 dict = 1.00 all = 0.3395 } ---------------------------- total value len = 10.391752 GB avg = 0.160 KB (by entry num) total key len = 1.789246 GB avg = 0.028 KB total ukey len = 1.246753 GB avg = 0.016 KB total ukey num = 0.077994195 Billion total entry num = 0.064801924 Billion write speed all = 0.199676913 MB/sec (with seq num) write speed all = 0.191178793 MB/sec (without seq num) 2024/11/24-01:51:55.761635 139957282305600 [compaction_job.cc:1948] [default] [JOB 39640] Generated table #287871: 20750 keys, 1000441 bytes, temperature: kWarm 2024/11/24-01:51:55.761680 139957282305600 EVENT_LOG_v1 {"time_micros": 1732384315761655, "cf_name": "default", "job": 39640, "event": "table_file_creation", "file_number": 287871, "file_size": 1000441, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4366001313, "largest_seqno": 4369139075, "table_properties": {"data_size": 876992, "index_size": 41280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 581000, "raw_average_key_size": 28, "raw_value_size": 2365500, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 20750, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732384315, "oldest_key_time": 0, "file_creation_time": 1732384315, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287871, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:51:55.771345 139957282305600 [compaction_job.cc:703] job-39640: subcompact[0], size: 0.489016 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39640/att-00 [287857,287858,287859,287860,287861,287862,287863,287864,287865,287866,287867,287868,287869,287870,287871] 2024/11/24-01:51:55.855519 139957282305600 (Original Log Time 2024/11/24-01:51:10.607155) EVENT_LOG_v1 {"time_micros": 1732384270607122, "job": 39640, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [287824], "files_L3": [287743, 287744, 287745, 287746, 287747, 287748, 287749, 287750, 287751, 287752, 287753, 287754, 287755, 287847], "score": 0.764633, "input_data_size": 509405265, "oldest_snapshot_seqno": 4381418007} 2024/11/24-01:51:55.855523 139957282305600 (Original Log Time 2024/11/24-01:51:55.853635) [compaction_job.cc:2022] [default] [JOB 39640] Compacted 1@2 + 14@3 files to L3 => 489015726 bytes 2024/11/24-01:51:55.855525 139957282305600 (Original Log Time 2024/11/24-01:51:55.854788) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.3 rd, 10.8 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(27.1, 458.7 +0.0 blob) out(466.4 +0.0 blob), read-write-amplify(35.1) write-amplify(17.2) OK, records in: 12556189, records dropped: 112429 output_compression: Snappy 2024/11/24-01:51:55.855532 139957282305600 (Original Log Time 2024/11/24-01:51:55.855498) EVENT_LOG_v1 {"time_micros": 1732384315854811, "cf": "default", "job": 39640, "event": "compaction_finished", "compaction_time_micros": 45164195, "compaction_time_cpu_micros": 18953734, "output_level": 3, "num_output_files": 15, "total_output_size": 489015726, "num_input_records": 12556189, "num_output_records": 12443760, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 333459076, "file_range_sync_nanos": 0, "file_fsync_nanos": 9953, "file_prepare_write_nanos": 34237, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/24-01:51:55.982402 139957282305600 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732384270,"executesMs":45383,"compactionJobId":39640,"attempt":0,"compactionInputRawBytes":1820415990,"compactionInputZipBytes":509405265,"compactionOutputRawBytes":0,"compactionOutputZipBytes":489015726,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-3ad6719e047bf51e06b178b868e5a284-df36f7a9c37261e5-00"}}, response = 2024/11/24-01:51:55.982499 139957282305600 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39640/att-00: olev 3, work 45.383 s, result 7.718 ms, install 1.920 ms, input{raw 1.695 GiB zip 485.807 MiB}