2024/11/23-14:30:10.643235 139957525694016 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00: fork to child time = 0.023521 sec 2024/11/23-14:30:10.644252 139957525694016 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00 2024/11/23-14:30:10.652632 139957525694016 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00 2024/11/23-14:30:10.653836 139957525694016 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:30:10.654180 139957525694016 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:30:10.708574 139957525694016 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27891/att-00/MANIFEST-206347 2024/11/23-14:30:10.742684 139957525694016 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27891/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 263413, last_sequence is 4025036198, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 263265 2024/11/23-14:30:10.742694 139957525694016 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:30:10.742900 139957525694016 [version_set.cc:5721] Creating manifest 263413 2024/11/23-14:30:10.778899 139957525694016 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:30:10.805571 139957525694016 [compaction_job.cc:2437] [default] [JOB 27891] Compacting 1@2 + 28@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-14:30:10.805595 139957525694016 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20841 Base level 2, inputs: [263384(27MB)], [263410(40MB) 263411(3389KB) 263353(35MB) 263354(34MB) 263355(35MB) 263356(34MB) 263357(33MB) 263358(34MB) 263359(34MB) 263360(34MB) 263361(34MB) 263362(34MB) 263363(34MB) 263364(35MB) 263365(37MB) 263366(33MB) 263367(35MB) 263368(37MB) 263369(34MB) 263370(35MB) 263371(37MB) 263372(35MB) 263373(36MB) 263374(35MB) 263375(35MB) 263376(36MB) 263377(41MB) 263396(39MB)] 2024/11/23-14:30:10.812858 139957525694016 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:30:10.812889 139957525694016 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:30:10.812908 139957525694016 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:30:10.812913 139957525694016 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:30:10.812926 139957525694016 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:30:11.456359 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263414.sst) /dev/shm/ToplingTemp/Topling-74407-BNqB2u 2024/11/23-14:30:12.411729 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263414.sst): first pass time = 0.96's, 140.495'MB/sec 2024/11/23-14:30:14.462858 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263414.sst): old prealloc_size = 147639500, real_size = 37813936 2024/11/23-14:30:14.463447 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263414.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930253 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0358 GB avg-val = 116.28 avg-zval = 38.51 2024/11/23-14:30:14.464321 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263414.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:15.528669 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263414.sst): second pass time = 2.05's, 52.761'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 39.446'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3086.652'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.23's, 181.298'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 41171.214'MB/sec zip my value time = 2.05's, unzip length = 0.108'GB zip my value throughput = 52.761'MB/sec zip pipeline throughput = 52.761'MB/sec entries = 930253 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.28 avg-zval = 38.51 usrkeys = 930253 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.28 avg-zval = 38.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930253 bytes = 3372176 } seq expand size = 6511771 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0358 GB dict = 1.08 MB all = 0.0420 GB } UnZip/Zip{ index = 10.5826 value = 3.0196 dict = 1.00 all = 3.1925 } Zip/UnZip{ index = 0.0945 value = 0.3312 dict = 1.00 all = 0.3132 } ---------------------------- total value len = 8.963302 GB avg = 0.168 KB (by entry num) total key len = 1.466868 GB avg = 0.028 KB total ukey len = 1.016483 GB avg = 0.015 KB total ukey num = 0.066480688 Billion total entry num = 0.053288417 Billion write speed all = 0.518828411 MB/sec (with seq num) write speed all = 0.497622587 MB/sec (without seq num) 2024/11/23-14:30:15.580598 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263414: 930253 keys, 42042190 bytes, temperature: kWarm 2024/11/23-14:30:15.580691 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343415580635, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263414, "file_size": 42042190, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 4024412263, "table_properties": {"data_size": 35823136, "index_size": 1758080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26047084, "raw_average_key_size": 28, "raw_value_size": 108170746, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930253, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343411, "oldest_key_time": 0, "file_creation_time": 1732343411, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263414, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:15.620816 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263415.sst) /dev/shm/ToplingTemp/Topling-74407-c2HKJv 2024/11/23-14:30:16.648408 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263415.sst): first pass time = 1.03's, 130.618'MB/sec 2024/11/23-14:30:19.021461 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263415.sst): old prealloc_size = 147639500, real_size = 32548768 2024/11/23-14:30:19.022045 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263415.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929630 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.38 avg-zval = 32.86 2024/11/23-14:30:19.022844 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263415.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:20.161416 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263415.sst): second pass time = 2.37's, 45.603'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.14's, 32.259'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3148.959'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.256'MB/sec write SST data time = 0.23's, 158.681'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33246.112'MB/sec zip my value time = 2.37's, unzip length = 0.108'GB zip my value throughput = 45.603'MB/sec zip pipeline throughput = 106.877'MB/sec entries = 929630 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.38 avg-zval = 32.86 usrkeys = 929630 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.38 avg-zval = 32.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929630 bytes = 3369920 } seq expand size = 6507410 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5143 value = 3.5416 dict = 1.00 all = 3.6500 } Zip/UnZip{ index = 0.0951 value = 0.2824 dict = 1.00 all = 0.2740 } ---------------------------- total value len = 9.077998 GB avg = 0.167 KB (by entry num) total key len = 1.492898 GB avg = 0.028 KB total ukey len = 1.035076 GB avg = 0.015 KB total ukey num = 0.067410318 Billion total entry num = 0.054218047 Billion write speed all = 0.525707366 MB/sec (with seq num) write speed all = 0.504136572 MB/sec (without seq num) 2024/11/23-14:30:20.213619 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263415: 929630 keys, 36771580 bytes, temperature: kWarm 2024/11/23-14:30:20.213716 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343420213662, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263415, "file_size": 36771580, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3636547395, "largest_seqno": 4012776090, "table_properties": {"data_size": 30547856, "index_size": 1768320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26029640, "raw_average_key_size": 28, "raw_value_size": 108188184, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929630, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343415, "oldest_key_time": 0, "file_creation_time": 1732343415, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263415, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:20.251728 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263416.sst) /dev/shm/ToplingTemp/Topling-74407-7rpSwx 2024/11/23-14:30:21.179722 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263416.sst): first pass time = 0.93's, 144.638'MB/sec 2024/11/23-14:30:22.681163 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263416.sst): old prealloc_size = 147639500, real_size = 32941152 2024/11/23-14:30:22.681680 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263416.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923900 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.27 avg-zval = 33.49 2024/11/23-14:30:22.682468 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263416.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:23.741232 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263416.sst): second pass time = 1.50's, 72.199'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 35.043'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3571.921'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.273'MB/sec write SST data time = 0.21's, 173.253'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 36618.701'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.199'MB/sec zip pipeline throughput = 72.199'MB/sec entries = 923900 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.27 avg-zval = 33.49 usrkeys = 923900 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.27 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923900 bytes = 3349152 } seq expand size = 6467300 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4434 value = 3.5018 dict = 1.00 all = 3.6131 } Zip/UnZip{ index = 0.0958 value = 0.2856 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 9.192814 GB avg = 0.167 KB (by entry num) total key len = 1.518767 GB avg = 0.028 KB total ukey len = 1.053554 GB avg = 0.015 KB total ukey num = 0.068334218 Billion total entry num = 0.055141947 Billion write speed all = 0.532609038 MB/sec (with seq num) write speed all = 0.510674573 MB/sec (without seq num) 2024/11/23-14:30:23.786883 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263416: 923900 keys, 37147150 bytes, temperature: kWarm 2024/11/23-14:30:23.786955 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343423786919, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263416, "file_size": 37147150, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4012776122, "table_properties": {"data_size": 30940624, "index_size": 1769344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25869200, "raw_average_key_size": 28, "raw_value_size": 108348624, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923900, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343420, "oldest_key_time": 0, "file_creation_time": 1732343420, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263416, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:23.824796 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263417.sst) /dev/shm/ToplingTemp/Topling-74407-OuSaJu 2024/11/23-14:30:24.728319 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263417.sst): first pass time = 0.90's, 148.555'MB/sec 2024/11/23-14:30:26.223376 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263417.sst): old prealloc_size = 147639500, real_size = 32803440 2024/11/23-14:30:26.223915 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263417.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931750 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.05 avg-zval = 33.04 2024/11/23-14:30:26.224718 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263417.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:27.275290 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263417.sst): second pass time = 1.49's, 72.358'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 35.205'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3466.215'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.14's, 270.366'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32011.465'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.358'MB/sec zip pipeline throughput = 72.358'MB/sec entries = 931750 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.05 avg-zval = 33.04 usrkeys = 931750 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.05 avg-zval = 33.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931750 bytes = 3377616 } seq expand size = 6522250 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4460 value = 3.5122 dict = 1.00 all = 3.6244 } Zip/UnZip{ index = 0.0957 value = 0.2847 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 9.307465 GB avg = 0.166 KB (by entry num) total key len = 1.544856 GB avg = 0.028 KB total ukey len = 1.072189 GB avg = 0.015 KB total ukey num = 0.069265968 Billion total entry num = 0.056073697 Billion write speed all = 0.539512216 MB/sec (with seq num) write speed all = 0.517211036 MB/sec (without seq num) 2024/11/23-14:30:27.321398 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263417: 931750 keys, 37031794 bytes, temperature: kWarm 2024/11/23-14:30:27.321463 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343427321429, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263417, "file_size": 37031794, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 4012776133, "table_properties": {"data_size": 30786400, "index_size": 1783936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26089000, "raw_average_key_size": 28, "raw_value_size": 108128864, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931750, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343423, "oldest_key_time": 0, "file_creation_time": 1732343423, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263417, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:27.358798 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263418.sst) /dev/shm/ToplingTemp/Topling-74407-YI8yax 2024/11/23-14:30:28.273261 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263418.sst): first pass time = 0.91's, 146.779'MB/sec 2024/11/23-14:30:29.763676 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263418.sst): old prealloc_size = 147639500, real_size = 32083120 2024/11/23-14:30:29.764235 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263418.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936354 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0301 GB avg-val = 115.34 avg-zval = 32.10 2024/11/23-14:30:29.765063 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263418.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:30.810392 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263418.sst): second pass time = 1.49's, 72.520'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 34.817'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3314.233'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.254'MB/sec write SST data time = 0.13's, 290.116'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 26529.244'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.520'MB/sec zip pipeline throughput = 72.520'MB/sec entries = 936354 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.34 avg-zval = 32.10 usrkeys = 936354 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.34 avg-zval = 32.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936354 bytes = 3511344 } seq expand size = 6554478 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4470 value = 3.5933 dict = 1.00 all = 3.6829 } Zip/UnZip{ index = 0.0957 value = 0.2783 dict = 1.00 all = 0.2715 } ---------------------------- total value len = 9.422019 GB avg = 0.165 KB (by entry num) total key len = 1.571074 GB avg = 0.028 KB total ukey len = 1.090916 GB avg = 0.016 KB total ukey num = 0.070202322 Billion total entry num = 0.057010051 Billion write speed all = 0.546414542 MB/sec (with seq num) write speed all = 0.523744948 MB/sec (without seq num) 2024/11/23-14:30:30.855229 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263418: 936354 keys, 36443046 bytes, temperature: kWarm 2024/11/23-14:30:30.855298 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343430855262, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263418, "file_size": 36443046, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012776149, "table_properties": {"data_size": 30056288, "index_size": 1792576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26217912, "raw_average_key_size": 28, "raw_value_size": 107999952, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936354, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343427, "oldest_key_time": 0, "file_creation_time": 1732343427, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263418, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:30.892144 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263419.sst) /dev/shm/ToplingTemp/Topling-74407-xhlCUu 2024/11/23-14:30:31.739175 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263419.sst): first pass time = 0.85's, 158.464'MB/sec 2024/11/23-14:30:33.236667 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263419.sst): old prealloc_size = 147639500, real_size = 30865216 2024/11/23-14:30:33.237225 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263419.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933773 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0289 GB avg-val = 115.74 avg-zval = 30.90 2024/11/23-14:30:33.238057 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263419.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:34.231472 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263419.sst): second pass time = 1.50's, 72.234'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 35.285'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3323.528'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.269'MB/sec write SST data time = 0.13's, 276.608'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38708.095'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.234'MB/sec zip pipeline throughput = 72.234'MB/sec entries = 933773 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.74 avg-zval = 30.90 usrkeys = 933773 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.74 avg-zval = 30.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933773 bytes = 3384944 } seq expand size = 6536411 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5139 value = 3.7453 dict = 1.00 all = 3.8238 } Zip/UnZip{ index = 0.0951 value = 0.2670 dict = 1.00 all = 0.2615 } ---------------------------- total value len = 9.536628 GB avg = 0.165 KB (by entry num) total key len = 1.597220 GB avg = 0.028 KB total ukey len = 1.109591 GB avg = 0.016 KB total ukey num = 0.071136095 Billion total entry num = 0.057943824 Billion write speed all = 0.553316676 MB/sec (with seq num) write speed all = 0.530279691 MB/sec (without seq num) 2024/11/23-14:30:34.284483 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263419: 933773 keys, 35100822 bytes, temperature: kWarm 2024/11/23-14:30:34.284558 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343434284516, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263419, "file_size": 35100822, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 4012776165, "table_properties": {"data_size": 28855344, "index_size": 1776256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26145644, "raw_average_key_size": 28, "raw_value_size": 108072122, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933773, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343430, "oldest_key_time": 0, "file_creation_time": 1732343430, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263419, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:34.322187 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263420.sst) /dev/shm/ToplingTemp/Topling-74407-aGcPxu 2024/11/23-14:30:35.223302 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263420.sst): first pass time = 0.90's, 148.952'MB/sec 2024/11/23-14:30:36.737900 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263420.sst): old prealloc_size = 147639500, real_size = 31926416 2024/11/23-14:30:36.738442 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263420.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930928 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0299 GB avg-val = 116.18 avg-zval = 32.13 2024/11/23-14:30:36.739219 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263420.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:37.755395 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263420.sst): second pass time = 1.51's, 71.463'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 35.417'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3415.572'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.262'MB/sec write SST data time = 0.13's, 282.939'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35356.239'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.463'MB/sec zip pipeline throughput = 71.463'MB/sec entries = 930928 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.18 avg-zval = 32.13 usrkeys = 930928 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.18 avg-zval = 32.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930928 bytes = 3258256 } seq expand size = 6516496 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4454 value = 3.6158 dict = 1.00 all = 3.7245 } Zip/UnZip{ index = 0.0957 value = 0.2766 dict = 1.00 all = 0.2685 } ---------------------------- total value len = 9.651296 GB avg = 0.164 KB (by entry num) total key len = 1.623286 GB avg = 0.028 KB total ukey len = 1.128210 GB avg = 0.016 KB total ukey num = 0.072067023 Billion total entry num = 0.058874752 Billion write speed all = 0.560212610 MB/sec (with seq num) write speed all = 0.536809611 MB/sec (without seq num) 2024/11/23-14:30:37.801044 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263420: 930928 keys, 36036340 bytes, temperature: kWarm 2024/11/23-14:30:37.801114 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343437801078, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263420, "file_size": 36036340, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776176, "table_properties": {"data_size": 29911040, "index_size": 1782464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26065984, "raw_average_key_size": 28, "raw_value_size": 108151768, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930928, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343434, "oldest_key_time": 0, "file_creation_time": 1732343434, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263420, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:37.838655 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263421.sst) /dev/shm/ToplingTemp/Topling-74407-GpqSBv 2024/11/23-14:30:38.803881 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263421.sst): first pass time = 0.97's, 139.059'MB/sec 2024/11/23-14:30:40.301953 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263421.sst): old prealloc_size = 147639500, real_size = 31463376 2024/11/23-14:30:40.302494 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263421.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932731 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0295 GB avg-val = 115.90 avg-zval = 31.58 2024/11/23-14:30:40.303192 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263421.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:41.136897 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263421.sst): second pass time = 1.50's, 72.205'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.043'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3573.478'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.330'MB/sec write SST data time = 0.13's, 272.452'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35483.198'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.205'MB/sec zip pipeline throughput = 72.205'MB/sec entries = 932731 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.90 avg-zval = 31.58 usrkeys = 932731 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.77 avg-zval = 31.58 TagRS{ kinds = RS_Key0_TagN bytes = 145840 } TagArray{ size = 781862 bytes = 2638800 } seq expand size = 5473034 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.5132 value = 3.6700 dict = 1.00 all = 3.8236 } Zip/UnZip{ index = 0.0951 value = 0.2725 dict = 1.00 all = 0.2615 } ---------------------------- total value len = 9.764871 GB avg = 0.163 KB (by entry num) total key len = 1.649402 GB avg = 0.028 KB total ukey len = 1.146864 GB avg = 0.016 KB total ukey num = 0.072999754 Billion total entry num = 0.059807483 Billion write speed all = 0.567058309 MB/sec (with seq num) write speed all = 0.543288539 MB/sec (without seq num) 2024/11/23-14:30:41.180682 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263421: 932731 keys, 35102294 bytes, temperature: kWarm 2024/11/23-14:30:41.180752 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343441180714, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263421, "file_size": 35102294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776192, "table_properties": {"data_size": 29455616, "index_size": 1774400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26116468, "raw_average_key_size": 28, "raw_value_size": 108101402, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932731, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343437, "oldest_key_time": 0, "file_creation_time": 1732343437, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263421, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:41.219962 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263422.sst) /dev/shm/ToplingTemp/Topling-74407-ifAc1w 2024/11/23-14:30:42.242056 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263422.sst): first pass time = 1.02's, 131.329'MB/sec 2024/11/23-14:30:43.634451 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263422.sst): old prealloc_size = 147639500, real_size = 34992944 2024/11/23-14:30:43.635096 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263422.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081594 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 96.09 avg-zval = 30.20 2024/11/23-14:30:43.635136 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263422.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:44.312522 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263422.sst): second pass time = 1.39's, 74.685'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.68's, 52.751'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3291.728'MB/sec (index lex order gen) rebuild zvType time = 0.54's, 0.500'MB/sec write SST data time = 0.14's, 261.681'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.034'MB, throughput = 31281.116'MB/sec zip my value time = 1.39's, unzip length = 0.104'GB zip my value throughput = 74.685'MB/sec zip pipeline throughput = 74.685'MB/sec entries = 1081594 avg-key = 28.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.20 usrkeys = 1081594 avg-key = 20.00 avg-zkey = 1.90 avg-val = 96.09 avg-zval = 30.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.03 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5161 value = 3.1818 dict = 1.00 all = 3.7525 } Zip/UnZip{ index = 0.0951 value = 0.3143 dict = 1.00 all = 0.2665 } ---------------------------- total value len = 9.868804 GB avg = 0.162 KB (by entry num) total key len = 1.679687 GB avg = 0.028 KB total ukey len = 1.168496 GB avg = 0.016 KB total ukey num = 0.074081348 Billion total entry num = 0.060889077 Billion write speed all = 0.573635717 MB/sec (with seq num) write speed all = 0.549439897 MB/sec (without seq num) 2024/11/23-14:30:44.356961 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263422: 1081594 keys, 35767344 bytes, temperature: kWarm 2024/11/23-14:30:44.357048 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343444357011, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263422, "file_size": 35767344, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32665312, "index_size": 2057024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30284632, "raw_average_key_size": 28, "raw_value_size": 103933141, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081594, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343441, "oldest_key_time": 0, "file_creation_time": 1732343441, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263422, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:44.398607 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263423.sst) /dev/shm/ToplingTemp/Topling-74407-qOx6Gt 2024/11/23-14:30:45.293909 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263423.sst): first pass time = 0.90's, 149.920'MB/sec 2024/11/23-14:30:46.591851 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263423.sst): old prealloc_size = 147639500, real_size = 32979920 2024/11/23-14:30:46.592445 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263423.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923868 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.28 avg-zval = 33.55 2024/11/23-14:30:46.593199 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263423.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:47.412537 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263423.sst): second pass time = 1.30's, 83.528'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.921'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3079.623'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 287.725'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32549.113'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.528'MB/sec zip pipeline throughput = 83.528'MB/sec entries = 923868 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.28 avg-zval = 33.55 usrkeys = 923868 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 33.55 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 767928 bytes = 2879744 } seq expand size = 5375496 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5134 value = 3.4961 dict = 1.00 all = 3.6408 } Zip/UnZip{ index = 0.0951 value = 0.2860 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 9.982529 GB avg = 0.161 KB (by entry num) total key len = 1.705555 GB avg = 0.028 KB total ukey len = 1.186974 GB avg = 0.016 KB total ukey num = 0.075005216 Billion total entry num = 0.061812945 Billion write speed all = 0.580480197 MB/sec (with seq num) write speed all = 0.555921036 MB/sec (without seq num) 2024/11/23-14:30:47.459433 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263423: 923868 keys, 36865058 bytes, temperature: kWarm 2024/11/23-14:30:47.459500 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343447459466, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263423, "file_size": 36865058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775885, "table_properties": {"data_size": 30991296, "index_size": 1757504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25868304, "raw_average_key_size": 28, "raw_value_size": 108349448, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923868, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343444, "oldest_key_time": 0, "file_creation_time": 1732343444, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263423, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:47.495215 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263424.sst) /dev/shm/ToplingTemp/Topling-74407-be31Yv 2024/11/23-14:30:48.433697 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263424.sst): first pass time = 0.94's, 143.023'MB/sec 2024/11/23-14:30:49.740413 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263424.sst): old prealloc_size = 147639500, real_size = 32821424 2024/11/23-14:30:49.741019 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263424.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930646 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.22 avg-zval = 33.10 2024/11/23-14:30:49.741812 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263424.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:50.597524 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263424.sst): second pass time = 1.31's, 82.819'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.223'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3083.499'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 297.033'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36183.613'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.819'MB/sec zip pipeline throughput = 82.819'MB/sec entries = 930646 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.22 avg-zval = 33.10 usrkeys = 930646 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.22 avg-zval = 33.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930646 bytes = 3373600 } seq expand size = 6514522 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4449 value = 3.5109 dict = 1.00 all = 3.6230 } Zip/UnZip{ index = 0.0957 value = 0.2848 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 10.097203 GB avg = 0.161 KB (by entry num) total key len = 1.731613 GB avg = 0.028 KB total ukey len = 1.205587 GB avg = 0.016 KB total ukey num = 0.075935862 Billion total entry num = 0.062743591 Billion write speed all = 0.587376652 MB/sec (with seq num) write speed all = 0.562451675 MB/sec (without seq num) 2024/11/23-14:30:50.642932 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263424: 930646 keys, 37046090 bytes, temperature: kWarm 2024/11/23-14:30:50.643025 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343450642967, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263424, "file_size": 37046090, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775902, "table_properties": {"data_size": 30806560, "index_size": 1782016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26058088, "raw_average_key_size": 28, "raw_value_size": 108159664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930646, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343447, "oldest_key_time": 0, "file_creation_time": 1732343447, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263424, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:50.677137 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263425.sst) /dev/shm/ToplingTemp/Topling-74407-fj0cqu 2024/11/23-14:30:51.703454 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263425.sst): first pass time = 1.03's, 130.780'MB/sec 2024/11/23-14:30:52.990113 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263425.sst): old prealloc_size = 147639500, real_size = 32722048 2024/11/23-14:30:52.990658 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263425.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932240 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 115.97 avg-zval = 32.95 2024/11/23-14:30:52.991427 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263425.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:53.849098 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263425.sst): second pass time = 1.29's, 84.083'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.884'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3396.758'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 293.355'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32192.778'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.083'MB/sec zip pipeline throughput = 84.083'MB/sec entries = 932240 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 32.95 usrkeys = 932240 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.97 avg-zval = 32.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932240 bytes = 3262848 } seq expand size = 6525680 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5115 value = 3.5199 dict = 1.00 all = 3.6437 } Zip/UnZip{ index = 0.0951 value = 0.2841 dict = 1.00 all = 0.2744 } ---------------------------- total value len = 10.211844 GB avg = 0.160 KB (by entry num) total key len = 1.757716 GB avg = 0.028 KB total ukey len = 1.224231 GB avg = 0.016 KB total ukey num = 0.076868102 Billion total entry num = 0.063675831 Billion write speed all = 0.594269517 MB/sec (with seq num) write speed all = 0.568978290 MB/sec (without seq num) 2024/11/23-14:30:53.894962 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263425: 932240 keys, 36835494 bytes, temperature: kWarm 2024/11/23-14:30:53.895042 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343453895007, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263425, "file_size": 36835494, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775918, "table_properties": {"data_size": 30715056, "index_size": 1773760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102720, "raw_average_key_size": 28, "raw_value_size": 108115060, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343450, "oldest_key_time": 0, "file_creation_time": 1732343450, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263425, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:53.931041 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263426.sst) /dev/shm/ToplingTemp/Topling-74407-QvF5jx 2024/11/23-14:30:54.840732 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263426.sst): first pass time = 0.91's, 147.549'MB/sec 2024/11/23-14:30:56.145360 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263426.sst): old prealloc_size = 147639500, real_size = 33262032 2024/11/23-14:30:56.145915 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263426.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928046 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.62 avg-zval = 33.69 2024/11/23-14:30:56.146752 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263426.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:30:57.006165 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263426.sst): second pass time = 1.30's, 83.036'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.675'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3346.163'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 291.558'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33508.677'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.036'MB/sec zip pipeline throughput = 83.036'MB/sec entries = 928046 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.62 avg-zval = 33.69 usrkeys = 928046 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.62 avg-zval = 33.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928046 bytes = 3480192 } seq expand size = 6496322 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5142 value = 3.4618 dict = 1.00 all = 3.5701 } Zip/UnZip{ index = 0.0951 value = 0.2889 dict = 1.00 all = 0.2801 } ---------------------------- total value len = 10.326572 GB avg = 0.160 KB (by entry num) total key len = 1.783701 GB avg = 0.028 KB total ukey len = 1.242792 GB avg = 0.016 KB total ukey num = 0.077796148 Billion total entry num = 0.064603877 Billion write speed all = 0.601161521 MB/sec (with seq num) write speed all = 0.575505707 MB/sec (without seq num) 2024/11/23-14:30:57.051369 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263426: 928046 keys, 37595396 bytes, temperature: kWarm 2024/11/23-14:30:57.051440 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343457051404, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263426, "file_size": 37595396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012774578, "table_properties": {"data_size": 31264512, "index_size": 1765312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25985288, "raw_average_key_size": 28, "raw_value_size": 108232528, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928046, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343453, "oldest_key_time": 0, "file_creation_time": 1732343453, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263426, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:30:57.087055 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263427.sst) /dev/shm/ToplingTemp/Topling-74407-jgmSxx 2024/11/23-14:30:58.153602 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263427.sst): first pass time = 1.07's, 125.849'MB/sec 2024/11/23-14:30:59.478901 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263427.sst): old prealloc_size = 147639500, real_size = 35013664 2024/11/23-14:30:59.479453 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263427.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928829 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0330 GB avg-val = 116.50 avg-zval = 35.53 2024/11/23-14:30:59.480245 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263427.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:00.322433 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263427.sst): second pass time = 1.32's, 81.724'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.512'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3365.737'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 309.129'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33074.764'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.724'MB/sec zip pipeline throughput = 81.724'MB/sec entries = 928829 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.50 avg-zval = 35.53 usrkeys = 928829 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.50 avg-zval = 35.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928829 bytes = 3367024 } seq expand size = 6501803 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.4451 value = 3.2788 dict = 1.00 all = 3.4210 } Zip/UnZip{ index = 0.0957 value = 0.3050 dict = 1.00 all = 0.2923 } ---------------------------- total value len = 10.441285 GB avg = 0.159 KB (by entry num) total key len = 1.809708 GB avg = 0.028 KB total ukey len = 1.261369 GB avg = 0.016 KB total ukey num = 0.078724977 Billion total entry num = 0.065532706 Billion write speed all = 0.608046834 MB/sec (with seq num) write speed all = 0.582026442 MB/sec (without seq num) 2024/11/23-14:31:00.369350 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263427: 928829 keys, 39233620 bytes, temperature: kWarm 2024/11/23-14:31:00.369421 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343460369384, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263427, "file_size": 39233620, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33002768, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26007212, "raw_average_key_size": 28, "raw_value_size": 108210614, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928829, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343457, "oldest_key_time": 0, "file_creation_time": 1732343457, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263427, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:00.407519 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263428.sst) /dev/shm/ToplingTemp/Topling-74407-WThD2u 2024/11/23-14:31:01.413706 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263428.sst): first pass time = 1.01's, 133.400'MB/sec 2024/11/23-14:31:02.682748 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263428.sst): old prealloc_size = 147639500, real_size = 30283552 2024/11/23-14:31:02.683302 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263428.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929511 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0283 GB avg-val = 116.40 avg-zval = 30.42 2024/11/23-14:31:02.684057 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263428.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:03.490895 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263428.sst): second pass time = 1.27's, 85.334'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.409'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3344.551'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.12's, 277.078'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34646.275'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.334'MB/sec zip pipeline throughput = 85.334'MB/sec entries = 929511 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.40 avg-zval = 30.42 usrkeys = 929511 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.40 avg-zval = 30.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929511 bytes = 3137120 } seq expand size = 6506577 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0343 GB } UnZip/Zip{ index = 10.4453 value = 3.8269 dict = 1.00 all = 3.9163 } Zip/UnZip{ index = 0.0957 value = 0.2613 dict = 1.00 all = 0.2553 } ---------------------------- total value len = 10.555983 GB avg = 0.159 KB (by entry num) total key len = 1.835735 GB avg = 0.028 KB total ukey len = 1.279959 GB avg = 0.016 KB total ukey num = 0.079654488 Billion total entry num = 0.066462217 Billion write speed all = 0.614934623 MB/sec (with seq num) write speed all = 0.588549310 MB/sec (without seq num) 2024/11/23-14:31:03.532063 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263428: 929511 keys, 34271682 bytes, temperature: kWarm 2024/11/23-14:31:03.532133 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343463532097, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263428, "file_size": 34271682, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3903523785, "largest_seqno": 4012775987, "table_properties": {"data_size": 28271184, "index_size": 1779776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26026308, "raw_average_key_size": 28, "raw_value_size": 108191442, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929511, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343460, "oldest_key_time": 0, "file_creation_time": 1732343460, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263428, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:03.567156 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263429.sst) /dev/shm/ToplingTemp/Topling-74407-2PZi1u 2024/11/23-14:31:04.588817 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263429.sst): first pass time = 1.02's, 131.377'MB/sec 2024/11/23-14:31:05.883046 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263429.sst): old prealloc_size = 147639500, real_size = 32740352 2024/11/23-14:31:05.883567 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263429.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929721 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.36 avg-zval = 33.06 2024/11/23-14:31:05.884343 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263429.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:06.721221 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263429.sst): second pass time = 1.29's, 83.642'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.101'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3515.401'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 293.895'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35555.804'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.642'MB/sec zip pipeline throughput = 83.642'MB/sec entries = 929721 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.36 avg-zval = 33.06 usrkeys = 929721 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.36 avg-zval = 33.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929721 bytes = 3370256 } seq expand size = 6508047 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5149 value = 3.5194 dict = 1.00 all = 3.6311 } Zip/UnZip{ index = 0.0951 value = 0.2841 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 10.670677 GB avg = 0.158 KB (by entry num) total key len = 1.861767 GB avg = 0.028 KB total ukey len = 1.298553 GB avg = 0.016 KB total ukey num = 0.080584209 Billion total entry num = 0.067391938 Billion write speed all = 0.621818417 MB/sec (with seq num) write speed all = 0.595068295 MB/sec (without seq num) 2024/11/23-14:31:06.766240 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263429: 929721 keys, 36963380 bytes, temperature: kWarm 2024/11/23-14:31:06.766310 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343466766274, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263429, "file_size": 36963380, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4012776021, "table_properties": {"data_size": 30739376, "index_size": 1768384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26032188, "raw_average_key_size": 28, "raw_value_size": 108185670, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929721, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343463, "oldest_key_time": 0, "file_creation_time": 1732343463, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263429, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:06.803622 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263430.sst) /dev/shm/ToplingTemp/Topling-74407-njfMHu 2024/11/23-14:31:07.689601 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263430.sst): first pass time = 0.89's, 151.498'MB/sec 2024/11/23-14:31:09.016052 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263430.sst): old prealloc_size = 147639500, real_size = 35693200 2024/11/23-14:31:09.016570 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263430.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923768 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0337 GB avg-val = 117.29 avg-zval = 36.47 2024/11/23-14:31:09.017268 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263430.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:09.824362 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263430.sst): second pass time = 1.33's, 81.736'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 48.840'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3580.058'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.339'MB/sec write SST data time = 0.13's, 311.375'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35467.597'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.736'MB/sec zip pipeline throughput = 81.736'MB/sec entries = 923768 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.29 avg-zval = 36.47 usrkeys = 923768 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.91 avg-zval = 36.47 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 740771 bytes = 2777904 } seq expand size = 5185397 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0337 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4446 value = 3.2159 dict = 1.00 all = 3.3999 } Zip/UnZip{ index = 0.0957 value = 0.3110 dict = 1.00 all = 0.2941 } ---------------------------- total value len = 10.784214 GB avg = 0.158 KB (by entry num) total key len = 1.887632 GB avg = 0.028 KB total ukey len = 1.317029 GB avg = 0.016 KB total ukey num = 0.081507977 Billion total entry num = 0.068315706 Billion write speed all = 0.628638353 MB/sec (with seq num) write speed all = 0.601525731 MB/sec (without seq num) 2024/11/23-14:31:09.871996 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263430: 923768 keys, 39476606 bytes, temperature: kWarm 2024/11/23-14:31:09.872063 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343469872026, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263430, "file_size": 39476606, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 33693184, "index_size": 1768896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25865504, "raw_average_key_size": 28, "raw_value_size": 108352288, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923768, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343466, "oldest_key_time": 0, "file_creation_time": 1732343466, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263430, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:09.906549 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263431.sst) /dev/shm/ToplingTemp/Topling-74407-Fu28fv 2024/11/23-14:31:10.895196 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263431.sst): first pass time = 0.99's, 135.765'MB/sec 2024/11/23-14:31:12.196554 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263431.sst): old prealloc_size = 147639500, real_size = 32405520 2024/11/23-14:31:12.197107 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263431.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.79 avg-zval = 32.56 2024/11/23-14:31:12.197892 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263431.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:13.064686 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263431.sst): second pass time = 1.30's, 83.102'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.206'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3407.704'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 288.736'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38308.430'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.102'MB/sec zip pipeline throughput = 83.102'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.79 avg-zval = 32.56 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 32.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5153 value = 3.5557 dict = 1.00 all = 3.6632 } Zip/UnZip{ index = 0.0951 value = 0.2812 dict = 1.00 all = 0.2730 } ---------------------------- total value len = 10.898830 GB avg = 0.157 KB (by entry num) total key len = 1.913769 GB avg = 0.028 KB total ukey len = 1.335698 GB avg = 0.016 KB total ukey num = 0.082441434 Billion total entry num = 0.069249163 Billion write speed all = 0.635518769 MB/sec (with seq num) write speed all = 0.608040102 MB/sec (without seq num) 2024/11/23-14:31:13.110555 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263431: 933457 keys, 36639654 bytes, temperature: kWarm 2024/11/23-14:31:13.110632 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343473110597, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263431, "file_size": 36639654, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 30396544, "index_size": 1775424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343469, "oldest_key_time": 0, "file_creation_time": 1732343469, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263431, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:13.156069 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263432.sst) /dev/shm/ToplingTemp/Topling-74407-ApfYCt 2024/11/23-14:31:13.997688 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263432.sst): first pass time = 0.84's, 159.484'MB/sec 2024/11/23-14:31:15.306276 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263432.sst): old prealloc_size = 147639500, real_size = 32545872 2024/11/23-14:31:15.306819 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263432.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934904 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0305 GB avg-val = 115.56 avg-zval = 32.65 2024/11/23-14:31:15.307625 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263432.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:16.156587 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263432.sst): second pass time = 1.31's, 82.612'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.262'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3437.061'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 289.037'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33872.545'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.612'MB/sec zip pipeline throughput = 82.612'MB/sec entries = 934904 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.56 avg-zval = 32.65 usrkeys = 934904 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.56 avg-zval = 32.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934904 bytes = 3389040 } seq expand size = 6544328 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4454 value = 3.5398 dict = 1.00 all = 3.6487 } Zip/UnZip{ index = 0.0957 value = 0.2825 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 11.013414 GB avg = 0.157 KB (by entry num) total key len = 1.939946 GB avg = 0.028 KB total ukey len = 1.354396 GB avg = 0.016 KB total ukey num = 0.083376338 Billion total entry num = 0.070184067 Billion write speed all = 0.642402205 MB/sec (with seq num) write speed all = 0.614556829 MB/sec (without seq num) 2024/11/23-14:31:16.202479 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263432: 934904 keys, 36785014 bytes, temperature: kWarm 2024/11/23-14:31:16.202549 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343476202512, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263432, "file_size": 36785014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30521856, "index_size": 1790080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26177312, "raw_average_key_size": 28, "raw_value_size": 108040512, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934904, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343473, "oldest_key_time": 0, "file_creation_time": 1732343473, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263432, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:16.244814 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263433.sst) /dev/shm/ToplingTemp/Topling-74407-mUFGuw 2024/11/23-14:31:17.197952 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263433.sst): first pass time = 0.95's, 140.822'MB/sec 2024/11/23-14:31:18.539893 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263433.sst): old prealloc_size = 147639500, real_size = 34927744 2024/11/23-14:31:18.540439 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263433.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0329 GB avg-val = 117.01 avg-zval = 35.57 2024/11/23-14:31:18.541269 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263433.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:19.385988 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263433.sst): second pass time = 1.34's, 80.750'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 46.396'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3387.469'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 309.227'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32163.432'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.750'MB/sec zip pipeline throughput = 80.750'MB/sec entries = 925568 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.01 avg-zval = 35.57 usrkeys = 925568 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.01 avg-zval = 35.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925568 bytes = 3470896 } seq expand size = 6478976 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4434 value = 3.2895 dict = 1.00 all = 3.4193 } Zip/UnZip{ index = 0.0958 value = 0.3040 dict = 1.00 all = 0.2925 } ---------------------------- total value len = 11.128195 GB avg = 0.156 KB (by entry num) total key len = 1.965862 GB avg = 0.028 KB total ukey len = 1.372907 GB avg = 0.016 KB total ukey num = 0.084301906 Billion total entry num = 0.071109635 Billion write speed all = 0.649275858 MB/sec (with seq num) write speed all = 0.621067783 MB/sec (without seq num) 2024/11/23-14:31:19.433856 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263433: 925568 keys, 39253506 bytes, temperature: kWarm 2024/11/23-14:31:19.433924 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343479433888, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263433, "file_size": 39253506, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 32923632, "index_size": 1772544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915904, "raw_average_key_size": 28, "raw_value_size": 108301824, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343476, "oldest_key_time": 0, "file_creation_time": 1732343476, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263433, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:19.472287 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263434.sst) /dev/shm/ToplingTemp/Topling-74407-BdKjNt 2024/11/23-14:31:20.490685 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263434.sst): first pass time = 1.02's, 131.797'MB/sec 2024/11/23-14:31:21.811530 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263434.sst): old prealloc_size = 147639500, real_size = 32414080 2024/11/23-14:31:21.812089 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263434.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.13 avg-zval = 32.65 2024/11/23-14:31:21.812959 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263434.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:22.684362 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263434.sst): second pass time = 1.32's, 81.915'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.981'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3354.475'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 281.230'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36826.253'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.915'MB/sec zip pipeline throughput = 81.915'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 32.65 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 32.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5112 value = 3.5563 dict = 1.00 all = 3.6630 } Zip/UnZip{ index = 0.0951 value = 0.2812 dict = 1.00 all = 0.2730 } ---------------------------- total value len = 11.242857 GB avg = 0.156 KB (by entry num) total key len = 1.991937 GB avg = 0.028 KB total ukey len = 1.391532 GB avg = 0.016 KB total ukey num = 0.085233148 Billion total entry num = 0.072040877 Billion write speed all = 0.656147035 MB/sec (with seq num) write speed all = 0.627574224 MB/sec (without seq num) 2024/11/23-14:31:22.730316 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263434: 931242 keys, 36641316 bytes, temperature: kWarm 2024/11/23-14:31:22.730376 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343482730343, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263434, "file_size": 36641316, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 30409200, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343479, "oldest_key_time": 0, "file_creation_time": 1732343479, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263434, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:22.772710 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263435.sst) /dev/shm/ToplingTemp/Topling-74407-ybmfCu 2024/11/23-14:31:23.692763 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263435.sst): first pass time = 0.92's, 145.885'MB/sec 2024/11/23-14:31:25.611408 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263435.sst): old prealloc_size = 147639500, real_size = 35092656 2024/11/23-14:31:25.611953 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263435.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0331 GB avg-val = 117.46 avg-zval = 35.87 2024/11/23-14:31:25.613496 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263435.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:26.558138 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263435.sst): second pass time = 1.92's, 56.510'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 41.630'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3375.656'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 306.589'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 28954.792'MB/sec zip my value time = 1.92's, unzip length = 0.108'GB zip my value throughput = 56.510'MB/sec zip pipeline throughput = 103.336'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.46 avg-zval = 35.87 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.46 avg-zval = 35.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4461 value = 3.2749 dict = 1.00 all = 3.4056 } Zip/UnZip{ index = 0.0957 value = 0.3054 dict = 1.00 all = 0.2936 } ---------------------------- total value len = 11.357698 GB avg = 0.156 KB (by entry num) total key len = 2.017773 GB avg = 0.028 KB total ukey len = 1.409986 GB avg = 0.016 KB total ukey num = 0.086155845 Billion total entry num = 0.072963574 Billion write speed all = 0.662994094 MB/sec (with seq num) write speed all = 0.634060880 MB/sec (without seq num) 2024/11/23-14:31:26.614423 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263435: 922697 keys, 39411006 bytes, temperature: kWarm 2024/11/23-14:31:26.614492 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343486614457, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263435, "file_size": 39411006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33095200, "index_size": 1766592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343482, "oldest_key_time": 0, "file_creation_time": 1732343482, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263435, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:26.654135 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263436.sst) /dev/shm/ToplingTemp/Topling-74407-kECDuv 2024/11/23-14:31:27.551207 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263436.sst): first pass time = 0.90's, 149.624'MB/sec 2024/11/23-14:31:30.384396 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263436.sst): old prealloc_size = 147639500, real_size = 32883584 2024/11/23-14:31:30.385100 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263436.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0309 GB avg-val = 116.15 avg-zval = 33.15 2024/11/23-14:31:30.386001 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263436.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:31.468941 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263436.sst): second pass time = 2.83's, 38.247'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 34.325'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2592.795'MB/sec (index lex order gen) rebuild zvType time = 0.93's, 0.251'MB/sec write SST data time = 0.16's, 239.919'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33246.108'MB/sec zip my value time = 2.83's, unzip length = 0.108'GB zip my value throughput = 38.247'MB/sec zip pipeline throughput = 84.030'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.15 avg-zval = 33.15 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.15 avg-zval = 33.15 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4434 value = 3.5036 dict = 1.00 all = 3.6054 } Zip/UnZip{ index = 0.0958 value = 0.2854 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 11.472363 GB avg = 0.155 KB (by entry num) total key len = 2.043843 GB avg = 0.028 KB total ukey len = 1.428608 GB avg = 0.016 KB total ukey num = 0.087086928 Billion total entry num = 0.073894657 Billion write speed all = 0.669807011 MB/sec (with seq num) write speed all = 0.640511714 MB/sec (without seq num) 2024/11/23-14:31:31.516587 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263436: 931083 keys, 37226772 bytes, temperature: kWarm 2024/11/23-14:31:31.516652 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343491516618, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263436, "file_size": 37226772, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30867504, "index_size": 1783104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343486, "oldest_key_time": 0, "file_creation_time": 1732343486, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263436, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:31.555702 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263437.sst) /dev/shm/ToplingTemp/Topling-74407-ThTYEx 2024/11/23-14:31:32.745858 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263437.sst): first pass time = 1.19's, 112.788'MB/sec 2024/11/23-14:31:34.619064 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263437.sst): old prealloc_size = 147639500, real_size = 34377520 2024/11/23-14:31:34.619787 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263437.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928022 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.63 avg-zval = 34.88 2024/11/23-14:31:34.620338 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263437.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:35.471413 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263437.sst): second pass time = 1.87's, 57.803'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.789'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2532.942'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.16's, 238.561'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33440.134'MB/sec zip my value time = 1.87's, unzip length = 0.108'GB zip my value throughput = 57.803'MB/sec zip pipeline throughput = 128.101'MB/sec entries = 928022 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.63 avg-zval = 34.88 usrkeys = 928022 avg-key = 20.00 avg-zkey = 1.92 avg-val = 120.81 avg-zval = 34.88 TagRS{ kinds = RS_Key0_TagN bytes = 145120 } TagArray{ size = 554366 bytes = 1940288 } seq expand size = 3880562 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4420 value = 3.3438 dict = 1.00 all = 3.5963 } Zip/UnZip{ index = 0.0958 value = 0.2991 dict = 1.00 all = 0.2781 } ---------------------------- total value len = 11.584477 GB avg = 0.155 KB (by entry num) total key len = 2.069828 GB avg = 0.028 KB total ukey len = 1.447168 GB avg = 0.016 KB total ukey num = 0.088014950 Billion total entry num = 0.074822679 Billion write speed all = 0.676516407 MB/sec (with seq num) write speed all = 0.646859081 MB/sec (without seq num) 2024/11/23-14:31:35.519091 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263437: 928022 keys, 37321439 bytes, temperature: kWarm 2024/11/23-14:31:35.519180 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343495519133, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263437, "file_size": 37321439, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 32367840, "index_size": 1777472, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25984616, "raw_average_key_size": 28, "raw_value_size": 108233156, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928022, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343491, "oldest_key_time": 0, "file_creation_time": 1732343491, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263437, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:35.598769 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263438.sst) /dev/shm/ToplingTemp/Topling-74407-sOgrst 2024/11/23-14:31:36.604608 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263438.sst): first pass time = 1.01's, 133.445'MB/sec 2024/11/23-14:31:38.255950 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263438.sst): old prealloc_size = 147639500, real_size = 36761904 2024/11/23-14:31:38.256541 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263438.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926650 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.84 avg-zval = 37.52 2024/11/23-14:31:38.256779 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263438.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:39.111781 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263438.sst): second pass time = 1.65's, 65.624'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 44.692'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3253.452'MB/sec (index lex order gen) rebuild zvType time = 0.62's, 0.373'MB/sec write SST data time = 0.23's, 163.164'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37837.851'MB/sec zip my value time = 1.65's, unzip length = 0.108'GB zip my value throughput = 65.624'MB/sec zip pipeline throughput = 65.624'MB/sec entries = 926650 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.84 avg-zval = 37.52 usrkeys = 926650 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.01 avg-zval = 37.52 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5152 value = 3.1142 dict = 1.00 all = 3.5092 } Zip/UnZip{ index = 0.0951 value = 0.3211 dict = 1.00 all = 0.2850 } ---------------------------- total value len = 11.693829 GB avg = 0.154 KB (by entry num) total key len = 2.095774 GB avg = 0.028 KB total ukey len = 1.465701 GB avg = 0.016 KB total ukey num = 0.088941600 Billion total entry num = 0.075749329 Billion write speed all = 0.683096708 MB/sec (with seq num) write speed all = 0.653077502 MB/sec (without seq num) 2024/11/23-14:31:39.161075 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263438: 926650 keys, 38247605 bytes, temperature: kWarm 2024/11/23-14:31:39.161171 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343499161120, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263438, "file_size": 38247605, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34767584, "index_size": 1762496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25946200, "raw_average_key_size": 28, "raw_value_size": 108271640, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926650, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343495, "oldest_key_time": 0, "file_creation_time": 1732343495, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263438, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:39.229461 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263439.sst) /dev/shm/ToplingTemp/Topling-74407-NE2Ext 2024/11/23-14:31:40.321968 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263439.sst): first pass time = 1.09's, 122.859'MB/sec 2024/11/23-14:31:41.870112 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263439.sst): old prealloc_size = 147639500, real_size = 38707616 2024/11/23-14:31:41.870646 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263439.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925132 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0367 GB avg-val = 117.08 avg-zval = 39.69 2024/11/23-14:31:41.871381 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263439.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:42.834881 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263439.sst): second pass time = 1.55's, 70.001'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 44.153'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3443.894'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.277'MB/sec write SST data time = 0.13's, 329.709'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31937.110'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 70.001'MB/sec zip pipeline throughput = 70.001'MB/sec entries = 925132 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 39.69 usrkeys = 925132 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.90 avg-zval = 39.69 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 769447 bytes = 2885440 } seq expand size = 5386129 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0367 GB dict = 1.08 MB all = 0.0426 GB } UnZip/Zip{ index = 10.5125 value = 2.9500 dict = 1.00 all = 3.1509 } Zip/UnZip{ index = 0.0951 value = 0.3390 dict = 1.00 all = 0.3174 } ---------------------------- total value len = 11.807529 GB avg = 0.154 KB (by entry num) total key len = 2.121677 GB avg = 0.028 KB total ukey len = 1.484204 GB avg = 0.017 KB total ukey num = 0.089866732 Billion total entry num = 0.076674461 Billion write speed all = 0.689885041 MB/sec (with seq num) write speed all = 0.659504811 MB/sec (without seq num) 2024/11/23-14:31:42.886501 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263439: 925132 keys, 42596640 bytes, temperature: kWarm 2024/11/23-14:31:42.886578 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343502886536, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263439, "file_size": 42596640, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4024412372, "table_properties": {"data_size": 36716112, "index_size": 1760064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903696, "raw_average_key_size": 28, "raw_value_size": 108314084, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925132, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343499, "oldest_key_time": 0, "file_creation_time": 1732343499, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263439, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:42.925707 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263440.sst) /dev/shm/ToplingTemp/Topling-74407-ayozSx 2024/11/23-14:31:43.914191 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263440.sst): first pass time = 0.99's, 135.788'MB/sec 2024/11/23-14:31:45.500753 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263440.sst): old prealloc_size = 147639500, real_size = 37610096 2024/11/23-14:31:45.501432 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263440.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923505 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0356 GB avg-val = 117.34 avg-zval = 38.57 2024/11/23-14:31:45.503530 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263440.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:31:46.517171 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263440.sst): second pass time = 1.58's, 68.366'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 40.573'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2668.846'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.272'MB/sec write SST data time = 0.17's, 246.593'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 37732.938'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.366'MB/sec zip pipeline throughput = 68.366'MB/sec entries = 923505 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.34 avg-zval = 38.57 usrkeys = 923505 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.34 avg-zval = 38.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923505 bytes = 2770528 } seq expand size = 6464535 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0356 GB dict = 1.08 MB all = 0.0412 GB } UnZip/Zip{ index = 10.5131 value = 3.0419 dict = 1.00 all = 3.2547 } Zip/UnZip{ index = 0.0951 value = 0.3287 dict = 1.00 all = 0.3072 } ---------------------------- total value len = 11.922354 GB avg = 0.154 KB (by entry num) total key len = 2.147536 GB avg = 0.028 KB total ukey len = 1.502674 GB avg = 0.017 KB total ukey num = 0.090790237 Billion total entry num = 0.077597966 Billion write speed all = 0.696725679 MB/sec (with seq num) write speed all = 0.665985141 MB/sec (without seq num) 2024/11/23-14:31:46.568808 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263440: 923505 keys, 41238158 bytes, temperature: kWarm 2024/11/23-14:31:46.568877 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343506568841, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263440, "file_size": 41238158, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777205, "largest_seqno": 4024412318, "table_properties": {"data_size": 35622176, "index_size": 1756864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25858140, "raw_average_key_size": 28, "raw_value_size": 108359590, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923505, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343502, "oldest_key_time": 0, "file_creation_time": 1732343502, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263440, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:46.606798 139957525694016 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263441.sst) /dev/shm/ToplingTemp/Topling-74407-IQrxAx 2024/11/23-14:31:46.723541 139957525694016 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263441.sst): first pass time = 0.12's, 238.650'MB/sec 2024/11/23-14:31:46.994134 139957525694016 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263441.sst): old prealloc_size = 147639500, real_size = 8246848 2024/11/23-14:31:46.994293 139957525694016 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263441.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 191942 prefix = 4 raw-key = 0.0031 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0225 GB zip-val = 0.0078 GB avg-val = 117.10 avg-zval = 40.81 2024/11/23-14:31:46.994723 139957525694016 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263441.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.028 GB 2024/11/23-14:31:47.216202 139957525694016 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00/263441.sst): second pass time = 0.27's, 83.417'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 40.591'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2638.406'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.277'MB/sec write SST data time = 0.05's, 185.256'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.235'MB, throughput = 9335.344'MB/sec zip my value time = 0.27's, unzip length = 0.022'GB zip my value throughput = 83.417'MB/sec zip pipeline throughput = 83.417'MB/sec entries = 191942 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.10 avg-zval = 40.81 usrkeys = 191942 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.10 avg-zval = 40.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 191942 bytes = 575840 } seq expand size = 1343594 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0038 GB value = 0.0225 GB dict = 0.24 MB all = 0.0279 GB } __ZipSize{ index = 0.0004 GB value = 0.0078 GB dict = 0.24 MB all = 0.0090 GB } UnZip/Zip{ index = 10.4754 value = 2.8697 dict = 1.00 all = 3.0904 } Zip/UnZip{ index = 0.0955 value = 0.3485 dict = 1.00 all = 0.3236 } ---------------------------- total value len = 11.946173 GB avg = 0.154 KB (by entry num) total key len = 2.152910 GB avg = 0.028 KB total ukey len = 1.506513 GB avg = 0.017 KB total ukey num = 0.090982179 Billion total entry num = 0.077789908 Billion write speed all = 0.698147182 MB/sec (with seq num) write speed all = 0.667331672 MB/sec (without seq num) 2024/11/23-14:31:47.236803 139957525694016 [compaction_job.cc:1948] [default] [JOB 27891] Generated table #263441: 191942 keys, 9012115 bytes, temperature: kWarm 2024/11/23-14:31:47.236890 139957525694016 EVENT_LOG_v1 {"time_micros": 1732343507236845, "cf_name": "default", "job": 27891, "event": "table_file_creation", "file_number": 263441, "file_size": 9012115, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777302, "largest_seqno": 4024412340, "table_properties": {"data_size": 7832240, "index_size": 366464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5374376, "raw_average_key_size": 28, "raw_value_size": 22476300, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 191942, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732343506, "oldest_key_time": 0, "file_creation_time": 1732343506, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 263441, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:31:47.252380 139957525694016 [compaction_job.cc:703] job-27891: subcompact[0], size: 1.024103 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27891/att-00 [263414,263415,263416,263417,263418,263419,263420,263421,263422,263423,263424,263425,263426,263427,263428,263429,263430,263431,263432,263433,263434,263435,263436,263437,263438,263439,263440,263441] 2024/11/23-14:31:47.567306 139957525694016 (Original Log Time 2024/11/23-14:30:10.805652) EVENT_LOG_v1 {"time_micros": 1732343410805611, "job": 27891, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [263384], "files_L3": [263410, 263411, 263353, 263354, 263355, 263356, 263357, 263358, 263359, 263360, 263361, 263362, 263363, 263364, 263365, 263366, 263367, 263368, 263369, 263370, 263371, 263372, 263373, 263374, 263375, 263376, 263377, 263396], "score": 0.992117, "input_data_size": 1046494078, "oldest_snapshot_seqno": 4025036178} 2024/11/23-14:31:47.567313 139957525694016 (Original Log Time 2024/11/23-14:31:47.565518) [compaction_job.cc:2022] [default] [JOB 27891] Compacted 1@2 + 28@3 files to L3 => 1024103111 bytes 2024/11/23-14:31:47.567316 139957525694016 (Original Log Time 2024/11/23-14:31:47.567188) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 28 0 0 0] max score 0.04, MB/sec: 10.9 rd, 10.6 wr, level 3, files in(1, 28) out(28 +0 blob) MB in(27.1, 970.9 +0.0 blob) out(976.7 +0.0 blob), read-write-amplify(72.8) write-amplify(36.0) OK, records in: 25543737, records dropped: 111993 output_compression: Snappy 2024/11/23-14:31:47.567324 139957525694016 (Original Log Time 2024/11/23-14:31:47.567269) EVENT_LOG_v1 {"time_micros": 1732343507567226, "cf": "default", "job": 27891, "event": "compaction_finished", "compaction_time_micros": 96446739, "compaction_time_cpu_micros": 40632564, "output_level": 3, "num_output_files": 28, "total_output_size": 1024103111, "num_input_records": 25543737, "num_output_records": 25431744, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 763736445, "file_range_sync_nanos": 0, "file_fsync_nanos": 19732, "file_prepare_write_nanos": 75072, "lsm_state": [0, 0, 0, 28, 0, 0, 0]} 2024/11/23-14:31:47.716339 139957525694016 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732343410,"executesMs":96914,"compactionJobId":27891,"attempt":0,"compactionInputRawBytes":3667634287,"compactionInputZipBytes":1046494078,"compactionOutputRawBytes":0,"compactionOutputZipBytes":1024103111,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-988b53a5f85b1f5587d9c3bf3d2cff4f-4c8cebf2f9219f5e-00"}}, response = 2024/11/23-14:31:47.716416 139957525694016 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27891/att-00: olev 3, work 96.914 s, result 7.800 ms, install 1.831 ms, input{raw 3.416 GiB zip 998.015 MiB}