2024/11/23-08:55:12.047083 139957450159680 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02: fork to child time = 0.006876 sec 2024/11/23-08:55:12.047423 139957450159680 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02 2024/11/23-08:55:12.056265 139957450159680 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02 2024/11/23-08:55:12.057173 139957450159680 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-08:55:12.057399 139957450159680 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-08:55:12.103867 139957450159680 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-23079/att-02/MANIFEST-206347 2024/11/23-08:55:12.133068 139957450159680 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-23079/att-02/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 253236, last_sequence is 3894522365, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 253076 2024/11/23-08:55:12.133074 139957450159680 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-08:55:12.133182 139957450159680 [version_set.cc:5721] Creating manifest 253236 2024/11/23-08:55:12.166804 139957450159680 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02: bottommost_level: fake = 1, rpc = 0 2024/11/23-08:55:12.189907 139957450159680 [compaction_job.cc:2437] [default] [JOB 23079] Compacting 1@2 + 18@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-08:55:12.189928 139957450159680 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 17332 Base level 2, inputs: [253171(27MB)], [253229(33MB) 253231(6348KB) 253136(34MB) 253137(35MB) 253138(35MB) 253139(8133KB) 253099(36MB) 253100(34MB) 253101(35MB) 253102(35MB) 253103(35MB) 253104(33MB) 253105(34MB) 253106(34MB) 253115(33MB) 253116(34MB) 253117(33MB) 253200(35MB)] 2024/11/23-08:55:12.194969 139957450159680 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-08:55:12.195000 139957450159680 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-08:55:12.195015 139957450159680 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-08:55:12.195020 139957450159680 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-08:55:12.195027 139957450159680 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-08:55:12.890605 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253237.sst) /dev/shm/ToplingTemp/Topling-6377-Li7tXp 2024/11/23-08:55:13.429011 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253237.sst): first pass time = 0.54's, 249.565'MB/sec 2024/11/23-08:55:14.681239 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253237.sst): old prealloc_size = 147639500, real_size = 30983200 2024/11/23-08:55:14.681736 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253237.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924739 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0290 GB avg-val = 117.14 avg-zval = 31.33 2024/11/23-08:55:14.682540 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253237.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:15.480212 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253237.sst): second pass time = 1.25's, 86.572'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 44.045'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3731.272'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.12's, 297.743'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33450.968'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.572'MB/sec zip pipeline throughput = 86.572'MB/sec entries = 924739 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.14 avg-zval = 31.33 usrkeys = 924739 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.14 avg-zval = 31.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924739 bytes = 3352192 } seq expand size = 6473173 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0290 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.3771 value = 3.7393 dict = 1.00 all = 3.8141 } Zip/UnZip{ index = 0.0964 value = 0.2674 dict = 1.00 all = 0.2622 } ---------------------------- total value len = 0.114798 GB avg = 0.124 KB (by entry num) total key len = 0.025893 GB avg = 0.028 KB total ukey len = 0.018495 GB avg = 0.020 KB total ukey num = 0.000924739 Billion total entry num = 0.000924739 Billion write speed all = 1.274574616 MB/sec (with seq num) write speed all = 1.207554005 MB/sec (without seq num) 2024/11/23-08:55:15.603319 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253237: 924739 keys, 35189576 bytes, temperature: kWarm 2024/11/23-08:55:15.603410 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323315603353, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253237, "file_size": 35189576, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3531372445, "largest_seqno": 3894100444, "table_properties": {"data_size": 28969552, "index_size": 1782272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25892692, "raw_average_key_size": 28, "raw_value_size": 108325046, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924739, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323312, "oldest_key_time": 0, "file_creation_time": 1732323312, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253237, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:15.635308 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253238.sst) /dev/shm/ToplingTemp/Topling-6377-yMa6po 2024/11/23-08:55:16.603824 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253238.sst): first pass time = 0.97's, 138.586'MB/sec 2024/11/23-08:55:17.856266 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253238.sst): old prealloc_size = 147639500, real_size = 31567040 2024/11/23-08:55:17.856771 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253238.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933836 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0296 GB avg-val = 115.73 avg-zval = 31.65 2024/11/23-08:55:17.857581 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253238.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:18.691669 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253238.sst): second pass time = 1.25's, 86.352'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.718'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3563.265'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.326'MB/sec write SST data time = 0.12's, 299.437'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36697.597'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.352'MB/sec zip pipeline throughput = 86.352'MB/sec entries = 933836 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.73 avg-zval = 31.65 usrkeys = 933836 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.73 avg-zval = 31.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933836 bytes = 3268448 } seq expand size = 6536852 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5135 value = 3.6563 dict = 1.00 all = 3.7611 } Zip/UnZip{ index = 0.0951 value = 0.2735 dict = 1.00 all = 0.2659 } ---------------------------- total value len = 0.229405 GB avg = 0.123 KB (by entry num) total key len = 0.052040 GB avg = 0.028 KB total ukey len = 0.037172 GB avg = 0.020 KB total ukey num = 0.001858575 Billion total entry num = 0.001858575 Billion write speed all = 2.477642003 MB/sec (with seq num) write speed all = 2.346749639 MB/sec (without seq num) 2024/11/23-08:55:18.736010 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253238: 933836 keys, 35686054 bytes, temperature: kWarm 2024/11/23-08:55:18.736078 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323318736043, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253238, "file_size": 35686054, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3894100460, "table_properties": {"data_size": 29556976, "index_size": 1776448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26147408, "raw_average_key_size": 28, "raw_value_size": 108070336, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933836, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323315, "oldest_key_time": 0, "file_creation_time": 1732323315, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253238, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:18.767596 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253239.sst) /dev/shm/ToplingTemp/Topling-6377-5ZI1vm 2024/11/23-08:55:19.783966 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253239.sst): first pass time = 1.02's, 132.061'MB/sec 2024/11/23-08:55:21.045883 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253239.sst): old prealloc_size = 147639500, real_size = 33746944 2024/11/23-08:55:21.046414 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253239.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923193 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.38 avg-zval = 34.39 2024/11/23-08:55:21.047228 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253239.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:21.894113 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253239.sst): second pass time = 1.26's, 85.940'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.743'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3460.021'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.12's, 317.320'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 22926.363'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.940'MB/sec zip pipeline throughput = 85.940'MB/sec entries = 923193 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.38 avg-zval = 34.39 usrkeys = 923193 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.38 avg-zval = 34.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923193 bytes = 3346592 } seq expand size = 6462351 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.4437 value = 3.4134 dict = 1.00 all = 3.5366 } Zip/UnZip{ index = 0.0958 value = 0.2930 dict = 1.00 all = 0.2828 } ---------------------------- total value len = 0.344236 GB avg = 0.124 KB (by entry num) total key len = 0.077890 GB avg = 0.028 KB total ukey len = 0.055635 GB avg = 0.020 KB total ukey num = 0.002781768 Billion total entry num = 0.002781768 Billion write speed all = 3.614196103 MB/sec (with seq num) write speed all = 3.423658406 MB/sec (without seq num) 2024/11/23-08:55:21.940260 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253239: 923193 keys, 37951418 bytes, temperature: kWarm 2024/11/23-08:55:21.940325 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323321940291, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253239, "file_size": 37951418, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3894100480, "table_properties": {"data_size": 31748016, "index_size": 1767936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25849404, "raw_average_key_size": 28, "raw_value_size": 108368342, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923193, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323318, "oldest_key_time": 0, "file_creation_time": 1732323318, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253239, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:21.971738 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253240.sst) /dev/shm/ToplingTemp/Topling-6377-bkcZLp 2024/11/23-08:55:22.993576 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253240.sst): first pass time = 1.02's, 131.354'MB/sec 2024/11/23-08:55:24.287442 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253240.sst): old prealloc_size = 147639500, real_size = 33013120 2024/11/23-08:55:24.287946 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253240.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929647 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.38 avg-zval = 33.36 2024/11/23-08:55:24.288758 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253240.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:25.134002 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253240.sst): second pass time = 1.29's, 83.668'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.987'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3666.396'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.12's, 313.181'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32059.042'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.668'MB/sec zip pipeline throughput = 83.668'MB/sec entries = 929647 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.38 avg-zval = 33.36 usrkeys = 929647 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.38 avg-zval = 33.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929647 bytes = 3369984 } seq expand size = 6507529 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5126 value = 3.4886 dict = 1.00 all = 3.6045 } Zip/UnZip{ index = 0.0951 value = 0.2866 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 0.458931 GB avg = 0.124 KB (by entry num) total key len = 0.103920 GB avg = 0.028 KB total ukey len = 0.074228 GB avg = 0.020 KB total ukey num = 0.003711415 Billion total entry num = 0.003711415 Billion write speed all = 4.689001650 MB/sec (with seq num) write speed all = 4.441649065 MB/sec (without seq num) 2024/11/23-08:55:25.178085 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253240: 929647 keys, 37236020 bytes, temperature: kWarm 2024/11/23-08:55:25.178150 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323325178116, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253240, "file_size": 37236020, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3621075956, "largest_seqno": 3894100492, "table_properties": {"data_size": 31011888, "index_size": 1768640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26030116, "raw_average_key_size": 28, "raw_value_size": 108187746, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929647, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323321, "oldest_key_time": 0, "file_creation_time": 1732323321, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253240, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:25.216314 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253241.sst) /dev/shm/ToplingTemp/Topling-6377-xAqxBp 2024/11/23-08:55:26.202261 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253241.sst): first pass time = 0.99's, 136.135'MB/sec 2024/11/23-08:55:27.470120 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253241.sst): old prealloc_size = 147639500, real_size = 32737904 2024/11/23-08:55:27.470623 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253241.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929471 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.40 avg-zval = 33.06 2024/11/23-08:55:27.471414 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253241.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:28.312897 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253241.sst): second pass time = 1.27's, 85.390'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.857'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3678.073'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 307.622'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33280.706'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.390'MB/sec zip pipeline throughput = 85.390'MB/sec entries = 929471 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.40 avg-zval = 33.06 usrkeys = 929471 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.40 avg-zval = 33.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929471 bytes = 3369344 } seq expand size = 6506297 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4459 value = 3.5212 dict = 1.00 all = 3.6314 } Zip/UnZip{ index = 0.0957 value = 0.2840 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 0.573630 GB avg = 0.124 KB (by entry num) total key len = 0.129945 GB avg = 0.028 KB total ukey len = 0.092818 GB avg = 0.020 KB total ukey num = 0.004640886 Billion total entry num = 0.004640886 Billion write speed all = 5.710126289 MB/sec (with seq num) write speed all = 5.408807533 MB/sec (without seq num) 2024/11/23-08:55:28.356365 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253241: 929471 keys, 36960044 bytes, temperature: kWarm 2024/11/23-08:55:28.356433 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323328356396, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253241, "file_size": 36960044, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3894100507, "table_properties": {"data_size": 30725792, "index_size": 1779584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26025188, "raw_average_key_size": 28, "raw_value_size": 108192578, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929471, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323325, "oldest_key_time": 0, "file_creation_time": 1732323325, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253241, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:28.392291 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253242.sst) /dev/shm/ToplingTemp/Topling-6377-q3Eq5o 2024/11/23-08:55:29.303787 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253242.sst): first pass time = 0.91's, 147.256'MB/sec 2024/11/23-08:55:30.570706 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253242.sst): old prealloc_size = 147639500, real_size = 32601920 2024/11/23-08:55:30.571232 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253242.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930406 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.26 avg-zval = 32.89 2024/11/23-08:55:30.572008 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253242.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:31.418337 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253242.sst): second pass time = 1.27's, 85.431'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.448'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3575.535'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.12's, 301.613'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34486.633'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.431'MB/sec zip pipeline throughput = 85.431'MB/sec entries = 930406 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.26 avg-zval = 32.89 usrkeys = 930406 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.26 avg-zval = 32.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930406 bytes = 3372736 } seq expand size = 6512842 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5116 value = 3.5350 dict = 1.00 all = 3.6446 } Zip/UnZip{ index = 0.0951 value = 0.2829 dict = 1.00 all = 0.2744 } ---------------------------- total value len = 0.688309 GB avg = 0.124 KB (by entry num) total key len = 0.155996 GB avg = 0.028 KB total ukey len = 0.111426 GB avg = 0.020 KB total ukey num = 0.005571292 Billion total entry num = 0.005571292 Billion write speed all = 6.683823041 MB/sec (with seq num) write speed all = 6.330988442 MB/sec (without seq num) 2024/11/23-08:55:31.463521 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253242: 930406 keys, 36826386 bytes, temperature: kWarm 2024/11/23-08:55:31.463587 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323331463552, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253242, "file_size": 36826386, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 3894100538, "table_properties": {"data_size": 30598896, "index_size": 1770240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26051368, "raw_average_key_size": 28, "raw_value_size": 108166364, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930406, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323328, "oldest_key_time": 0, "file_creation_time": 1732323328, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253242, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:31.503276 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253243.sst) /dev/shm/ToplingTemp/Topling-6377-qgSHfq 2024/11/23-08:55:32.509913 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253243.sst): first pass time = 1.01's, 133.338'MB/sec 2024/11/23-08:55:33.771286 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253243.sst): old prealloc_size = 147639500, real_size = 31990208 2024/11/23-08:55:33.771800 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253243.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931468 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0300 GB avg-val = 116.09 avg-zval = 32.19 2024/11/23-08:55:33.773363 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253243.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:34.605462 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253243.sst): second pass time = 1.26's, 85.782'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.279'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3585.634'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.12's, 302.428'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38792.097'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.782'MB/sec zip pipeline throughput = 85.782'MB/sec entries = 931468 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 32.19 usrkeys = 931468 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 32.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931468 bytes = 3260160 } seq expand size = 6520276 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5149 value = 3.6063 dict = 1.00 all = 3.7178 } Zip/UnZip{ index = 0.0951 value = 0.2773 dict = 1.00 all = 0.2690 } ---------------------------- total value len = 0.802966 GB avg = 0.123 KB (by entry num) total key len = 0.182077 GB avg = 0.028 KB total ukey len = 0.130055 GB avg = 0.020 KB total ukey num = 0.006502760 Billion total entry num = 0.006502760 Billion write speed all = 7.606051757 MB/sec (with seq num) write speed all = 7.204361331 MB/sec (without seq num) 2024/11/23-08:55:34.648585 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253243: 931468 keys, 36101292 bytes, temperature: kWarm 2024/11/23-08:55:34.648649 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323334648615, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253243, "file_size": 36101292, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 3894100549, "table_properties": {"data_size": 29985456, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26081104, "raw_average_key_size": 28, "raw_value_size": 108136712, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931468, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323331, "oldest_key_time": 0, "file_creation_time": 1732323331, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253243, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:34.681330 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253244.sst) /dev/shm/ToplingTemp/Topling-6377-cDdKNm 2024/11/23-08:55:35.550287 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253244.sst): first pass time = 0.87's, 154.464'MB/sec 2024/11/23-08:55:36.800400 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253244.sst): old prealloc_size = 147639500, real_size = 31195584 2024/11/23-08:55:36.800910 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253244.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932245 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0292 GB avg-val = 115.97 avg-zval = 31.31 2024/11/23-08:55:36.801669 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253244.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:37.623759 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253244.sst): second pass time = 1.25's, 86.540'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.744'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3611.040'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.12's, 286.942'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38305.156'MB/sec zip my value time = 1.25's, unzip length = 0.108'GB zip my value throughput = 86.540'MB/sec zip pipeline throughput = 86.540'MB/sec entries = 932245 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 31.31 usrkeys = 932245 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.97 avg-zval = 31.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932245 bytes = 3146336 } seq expand size = 6525715 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.5145 value = 3.7039 dict = 1.00 all = 3.8138 } Zip/UnZip{ index = 0.0951 value = 0.2700 dict = 1.00 all = 0.2622 } ---------------------------- total value len = 0.917607 GB avg = 0.123 KB (by entry num) total key len = 0.208180 GB avg = 0.028 KB total ukey len = 0.148700 GB avg = 0.020 KB total ukey num = 0.007435005 Billion total entry num = 0.007435005 Billion write speed all = 8.494828165 MB/sec (with seq num) write speed all = 8.046010920 MB/sec (without seq num) 2024/11/23-08:55:37.666502 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253244: 932245 keys, 35192726 bytes, temperature: kWarm 2024/11/23-08:55:37.666569 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323337666534, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253244, "file_size": 35192726, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 3894100568, "table_properties": {"data_size": 29189104, "index_size": 1773248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102860, "raw_average_key_size": 28, "raw_value_size": 108114922, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932245, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323334, "oldest_key_time": 0, "file_creation_time": 1732323334, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253244, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:37.700440 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253245.sst) /dev/shm/ToplingTemp/Topling-6377-p8noDo 2024/11/23-08:55:38.737239 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253245.sst): first pass time = 1.04's, 129.458'MB/sec 2024/11/23-08:55:40.066383 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253245.sst): old prealloc_size = 147639500, real_size = 35052928 2024/11/23-08:55:40.066971 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253245.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081740 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1039 GB zip-val = 0.0327 GB avg-val = 96.08 avg-zval = 30.23 2024/11/23-08:55:40.067141 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253245.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:40.739682 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253245.sst): second pass time = 1.33's, 78.247'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.67's, 53.488'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3548.975'MB/sec (index lex order gen) rebuild zvType time = 0.54's, 0.500'MB/sec write SST data time = 0.13's, 273.781'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.032'MB, throughput = 22785.973'MB/sec zip my value time = 1.33's, unzip length = 0.104'GB zip my value throughput = 78.247'MB/sec zip pipeline throughput = 78.247'MB/sec entries = 1081740 avg-key = 28.00 avg-zkey = 1.93 avg-val = 96.08 avg-zval = 30.23 usrkeys = 1081740 avg-key = 20.00 avg-zkey = 1.93 avg-val = 96.12 avg-zval = 30.23 TagRS{ kinds = RS_Key0_TagN bytes = 169120 } TagArray{ size = 6880 bytes = 18080 } seq expand size = 48160 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0327 GB dict = 1.03 MB all = 0.0360 GB } UnZip/Zip{ index = 10.3799 value = 3.1785 dict = 1.00 all = 3.7270 } Zip/UnZip{ index = 0.0963 value = 0.3146 dict = 1.00 all = 0.2683 } ---------------------------- total value len = 1.021584 GB avg = 0.120 KB (by entry num) total key len = 0.238469 GB avg = 0.028 KB total ukey len = 0.170335 GB avg = 0.020 KB total ukey num = 0.008516745 Billion total entry num = 0.008516745 Billion write speed all = 9.289542394 MB/sec (with seq num) write speed all = 8.787235563 MB/sec (without seq num) 2024/11/23-08:55:40.784555 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253245: 1081740 keys, 36012144 bytes, temperature: kWarm 2024/11/23-08:55:40.784618 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323340784584, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253245, "file_size": 36012144, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3894098809, "table_properties": {"data_size": 32698032, "index_size": 2084288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30288720, "raw_average_key_size": 28, "raw_value_size": 103929061, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081740, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323337, "oldest_key_time": 0, "file_creation_time": 1732323337, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253245, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:40.815671 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253246.sst) /dev/shm/ToplingTemp/Topling-6377-CMX5Vm 2024/11/23-08:55:41.709542 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253246.sst): first pass time = 0.89's, 150.161'MB/sec 2024/11/23-08:55:43.007105 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253246.sst): old prealloc_size = 147639500, real_size = 34578976 2024/11/23-08:55:43.007609 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253246.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921869 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0326 GB avg-val = 117.59 avg-zval = 35.36 2024/11/23-08:55:43.008235 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253246.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:43.811147 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253246.sst): second pass time = 1.30's, 83.596'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 47.097'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3650.581'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.339'MB/sec write SST data time = 0.12's, 306.912'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32161.962'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.596'MB/sec zip pipeline throughput = 83.596'MB/sec entries = 921869 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.59 avg-zval = 35.36 usrkeys = 921869 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.21 avg-zval = 35.36 TagRS{ kinds = RS_Key0_TagN bytes = 144160 } TagArray{ size = 608326 bytes = 2281232 } seq expand size = 4258282 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5125 value = 3.3259 dict = 1.00 all = 3.5445 } Zip/UnZip{ index = 0.0951 value = 0.3007 dict = 1.00 all = 0.2821 } ---------------------------- total value len = 1.134248 GB avg = 0.120 KB (by entry num) total key len = 0.264281 GB avg = 0.028 KB total ukey len = 0.188772 GB avg = 0.020 KB total ukey num = 0.009438614 Billion total entry num = 0.009438614 Billion write speed all = 10.082136376 MB/sec (with seq num) write speed all = 9.537785094 MB/sec (without seq num) 2024/11/23-08:55:43.860069 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253246: 921869 keys, 37866944 bytes, temperature: kWarm 2024/11/23-08:55:43.860132 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323343860098, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253246, "file_size": 37866944, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3853794432, "table_properties": {"data_size": 32594448, "index_size": 1753856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25812332, "raw_average_key_size": 28, "raw_value_size": 108405398, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921869, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323340, "oldest_key_time": 0, "file_creation_time": 1732323340, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253246, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:43.899743 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253247.sst) /dev/shm/ToplingTemp/Topling-6377-V99s9p 2024/11/23-08:55:44.914570 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253247.sst): first pass time = 1.01's, 132.262'MB/sec 2024/11/23-08:55:46.177549 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253247.sst): old prealloc_size = 147639500, real_size = 33044896 2024/11/23-08:55:46.178121 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253247.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929337 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.42 avg-zval = 33.40 2024/11/23-08:55:46.178945 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253247.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:47.027313 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253247.sst): second pass time = 1.26's, 85.727'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.856'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3200.927'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.12's, 307.699'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33602.939'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.727'MB/sec zip pipeline throughput = 85.727'MB/sec entries = 929337 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.42 avg-zval = 33.40 usrkeys = 929337 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.42 avg-zval = 33.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929337 bytes = 3368864 } seq expand size = 6505359 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5129 value = 3.4852 dict = 1.00 all = 3.6016 } Zip/UnZip{ index = 0.0951 value = 0.2869 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 1.248950 GB avg = 0.120 KB (by entry num) total key len = 0.290303 GB avg = 0.028 KB total ukey len = 0.207359 GB avg = 0.020 KB total ukey num = 0.010367951 Billion total entry num = 0.010367951 Billion write speed all = 10.845171001 MB/sec (with seq num) write speed all = 10.260771946 MB/sec (without seq num) 2024/11/23-08:55:47.072694 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253247: 929337 keys, 37266576 bytes, temperature: kWarm 2024/11/23-08:55:47.072762 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323347072726, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253247, "file_size": 37266576, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3894100018, "table_properties": {"data_size": 31044368, "index_size": 1768000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26021436, "raw_average_key_size": 28, "raw_value_size": 108196370, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929337, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323343, "oldest_key_time": 0, "file_creation_time": 1732323343, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253247, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:47.107639 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253248.sst) /dev/shm/ToplingTemp/Topling-6377-e1RTCp 2024/11/23-08:55:48.049204 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253248.sst): first pass time = 0.94's, 142.554'MB/sec 2024/11/23-08:55:49.335685 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253248.sst): old prealloc_size = 147639500, real_size = 33123888 2024/11/23-08:55:49.336226 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253248.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929283 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.43 avg-zval = 33.49 2024/11/23-08:55:49.336998 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253248.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:50.189456 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253248.sst): second pass time = 1.29's, 84.153'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.470'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3388.219'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 302.696'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29760.172'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.153'MB/sec zip pipeline throughput = 84.153'MB/sec entries = 929283 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.43 avg-zval = 33.49 usrkeys = 929283 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.43 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929283 bytes = 3136352 } seq expand size = 6504981 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5123 value = 3.4764 dict = 1.00 all = 3.6164 } Zip/UnZip{ index = 0.0951 value = 0.2877 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 1.363653 GB avg = 0.121 KB (by entry num) total key len = 0.316323 GB avg = 0.028 KB total ukey len = 0.225945 GB avg = 0.020 KB total ukey num = 0.011297234 Billion total entry num = 0.011297234 Billion write speed all = 11.578698344 MB/sec (with seq num) write speed all = 10.955797389 MB/sec (without seq num) 2024/11/23-08:55:50.235358 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253248: 929283 keys, 37113324 bytes, temperature: kWarm 2024/11/23-08:55:50.235425 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323350235390, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253248, "file_size": 37113324, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 3894099878, "table_properties": {"data_size": 31123360, "index_size": 1768000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26019924, "raw_average_key_size": 28, "raw_value_size": 108197894, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929283, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323347, "oldest_key_time": 0, "file_creation_time": 1732323347, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253248, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:50.266215 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253249.sst) /dev/shm/ToplingTemp/Topling-6377-DBMWRp 2024/11/23-08:55:51.261970 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253249.sst): first pass time = 1.00's, 134.795'MB/sec 2024/11/23-08:55:52.524956 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253249.sst): old prealloc_size = 147639500, real_size = 33242880 2024/11/23-08:55:52.525476 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253249.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929390 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.41 avg-zval = 33.62 2024/11/23-08:55:52.526312 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253249.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:53.380353 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253249.sst): second pass time = 1.26's, 85.715'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.935'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3682.289'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.12's, 304.877'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32700.370'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.715'MB/sec zip pipeline throughput = 85.715'MB/sec entries = 929390 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.41 avg-zval = 33.62 usrkeys = 929390 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.41 avg-zval = 33.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929390 bytes = 3485232 } seq expand size = 6505730 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5116 value = 3.4631 dict = 1.00 all = 3.5715 } Zip/UnZip{ index = 0.0951 value = 0.2888 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 1.478353 GB avg = 0.121 KB (by entry num) total key len = 0.342345 GB avg = 0.028 KB total ukey len = 0.244532 GB avg = 0.020 KB total ukey num = 0.012226624 Billion total entry num = 0.012226624 Billion write speed all = 12.278558046 MB/sec (with seq num) write speed all = 11.618919803 MB/sec (without seq num) 2024/11/23-08:55:53.426064 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253249: 929390 keys, 37580500 bytes, temperature: kWarm 2024/11/23-08:55:53.426131 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323353426097, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253249, "file_size": 37580500, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3894100603, "table_properties": {"data_size": 31242032, "index_size": 1768320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26022920, "raw_average_key_size": 28, "raw_value_size": 108194924, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929390, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323350, "oldest_key_time": 0, "file_creation_time": 1732323350, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253249, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:53.466526 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253250.sst) /dev/shm/ToplingTemp/Topling-6377-nJRR5m 2024/11/23-08:55:54.448363 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253250.sst): first pass time = 0.98's, 136.708'MB/sec 2024/11/23-08:55:55.723415 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253250.sst): old prealloc_size = 147639500, real_size = 33120384 2024/11/23-08:55:55.723918 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253250.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929642 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.38 avg-zval = 33.47 2024/11/23-08:55:55.725882 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253250.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:56.582332 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253250.sst): second pass time = 1.27's, 84.897'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.343'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3664.489'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.12's, 298.104'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34958.995'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.897'MB/sec zip pipeline throughput = 84.897'MB/sec entries = 929642 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.38 avg-zval = 33.47 usrkeys = 929642 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.38 avg-zval = 33.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929642 bytes = 3253760 } seq expand size = 6507494 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5125 value = 3.4766 dict = 1.00 all = 3.6054 } Zip/UnZip{ index = 0.0951 value = 0.2876 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 1.593049 GB avg = 0.121 KB (by entry num) total key len = 0.368375 GB avg = 0.028 KB total ukey len = 0.263125 GB avg = 0.020 KB total ukey num = 0.013156266 Billion total entry num = 0.013156266 Billion write speed all = 12.947995696 MB/sec (with seq num) write speed all = 12.253205482 MB/sec (without seq num) 2024/11/23-08:55:56.627143 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253250: 929642 keys, 37226924 bytes, temperature: kWarm 2024/11/23-08:55:56.627207 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323356627171, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253250, "file_size": 37226924, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3894100268, "table_properties": {"data_size": 31119152, "index_size": 1768640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26029976, "raw_average_key_size": 28, "raw_value_size": 108187824, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929642, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323353, "oldest_key_time": 0, "file_creation_time": 1732323353, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253250, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:56.663057 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253251.sst) /dev/shm/ToplingTemp/Topling-6377-DwD5Yp 2024/11/23-08:55:57.655940 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253251.sst): first pass time = 0.99's, 135.187'MB/sec 2024/11/23-08:55:58.935725 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253251.sst): old prealloc_size = 147639500, real_size = 33121760 2024/11/23-08:55:58.936247 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253251.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929357 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.42 avg-zval = 33.49 2024/11/23-08:55:58.938348 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253251.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:55:59.792913 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253251.sst): second pass time = 1.28's, 84.594'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.566'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3517.771'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.12's, 299.175'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 37044.637'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.594'MB/sec zip pipeline throughput = 84.594'MB/sec entries = 929357 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.42 avg-zval = 33.49 usrkeys = 929357 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.42 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929357 bytes = 3368928 } seq expand size = 6505499 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5112 value = 3.4766 dict = 1.00 all = 3.5942 } Zip/UnZip{ index = 0.0951 value = 0.2876 dict = 1.00 all = 0.2782 } ---------------------------- total value len = 1.707750 GB avg = 0.121 KB (by entry num) total key len = 0.394397 GB avg = 0.028 KB total ukey len = 0.281712 GB avg = 0.020 KB total ukey num = 0.014085623 Billion total entry num = 0.014085623 Billion write speed all = 13.588950750 MB/sec (with seq num) write speed all = 12.860519083 MB/sec (without seq num) 2024/11/23-08:55:59.838226 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253251: 929357 keys, 37343032 bytes, temperature: kWarm 2024/11/23-08:55:59.838292 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323359838257, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253251, "file_size": 37343032, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3894100336, "table_properties": {"data_size": 31120912, "index_size": 1768320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26021996, "raw_average_key_size": 28, "raw_value_size": 108195850, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929357, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323356, "oldest_key_time": 0, "file_creation_time": 1732323356, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253251, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:55:59.873192 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253252.sst) /dev/shm/ToplingTemp/Topling-6377-Txj0rp 2024/11/23-08:56:00.719006 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253252.sst): first pass time = 0.85's, 158.693'MB/sec 2024/11/23-08:56:01.980790 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253252.sst): old prealloc_size = 147639500, real_size = 31959456 2024/11/23-08:56:01.981324 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253252.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933383 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0300 GB avg-val = 115.80 avg-zval = 32.09 2024/11/23-08:56:01.982171 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253252.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-08:56:02.827102 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253252.sst): second pass time = 1.26's, 85.712'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.768'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3455.121'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.12's, 295.611'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33378.638'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.712'MB/sec zip pipeline throughput = 85.712'MB/sec entries = 933383 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.80 avg-zval = 32.09 usrkeys = 933383 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.80 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933383 bytes = 3383536 } seq expand size = 6533681 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5130 value = 3.6088 dict = 1.00 all = 3.7083 } Zip/UnZip{ index = 0.0951 value = 0.2771 dict = 1.00 all = 0.2697 } ---------------------------- total value len = 1.822367 GB avg = 0.121 KB (by entry num) total key len = 0.420532 GB avg = 0.028 KB total ukey len = 0.300380 GB avg = 0.020 KB total ukey num = 0.015019006 Billion total entry num = 0.015019006 Billion write speed all = 14.219904278 MB/sec (with seq num) write speed all = 13.458144263 MB/sec (without seq num) 2024/11/23-08:56:02.871743 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253252: 933383 keys, 36193602 bytes, temperature: kWarm 2024/11/23-08:56:02.871809 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323362871775, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253252, "file_size": 36193602, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 3894099945, "table_properties": {"data_size": 29950224, "index_size": 1775680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26134724, "raw_average_key_size": 28, "raw_value_size": 108083042, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933383, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323359, "oldest_key_time": 0, "file_creation_time": 1732323359, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253252, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:56:02.903252 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253253.sst) /dev/shm/ToplingTemp/Topling-6377-y0gLHn 2024/11/23-08:56:03.187293 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253253.sst): first pass time = 0.28's, 244.849'MB/sec 2024/11/23-08:56:03.861690 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253253.sst): old prealloc_size = 147639500, real_size = 18305504 2024/11/23-08:56:03.862007 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253253.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 482805 prefix = 4 raw-key = 0.0077 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0560 GB zip-val = 0.0173 GB avg-val = 116.03 avg-zval = 35.76 2024/11/23-08:56:03.862434 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253253.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0011 GB, waited 0.000 sec, Key+Value = 0.070 GB 2024/11/23-08:56:04.294660 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02/253253.sst): second pass time = 0.67's, 83.152'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.43's, 47.350'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3098.876'MB/sec (index lex order gen) rebuild zvType time = 0.37's, 0.327'MB/sec write SST data time = 0.06's, 321.810'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.561'MB, throughput = 19371.924'MB/sec zip my value time = 0.67's, unzip length = 0.056'GB zip my value throughput = 83.152'MB/sec zip pipeline throughput = 83.152'MB/sec entries = 482805 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.03 avg-zval = 35.76 usrkeys = 482805 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.03 avg-zval = 35.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 482805 bytes = 1750176 } seq expand size = 3379635 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0097 GB value = 0.0560 GB dict = 0.56 MB all = 0.0695 GB } __ZipSize{ index = 0.0009 GB value = 0.0173 GB dict = 0.56 MB all = 0.0205 GB } UnZip/Zip{ index = 10.5045 value = 3.2446 dict = 1.00 all = 3.3922 } Zip/UnZip{ index = 0.0952 value = 0.3082 dict = 1.00 all = 0.2948 } ---------------------------- total value len = 1.881766 GB avg = 0.121 KB (by entry num) total key len = 0.434051 GB avg = 0.028 KB total ukey len = 0.310036 GB avg = 0.020 KB total ukey num = 0.015501811 Billion total entry num = 0.015501811 Billion write speed all = 14.546854208 MB/sec (with seq num) write speed all = 13.767854464 MB/sec (without seq num) 2024/11/23-08:56:04.323013 139957450159680 [compaction_job.cc:1948] [default] [JOB 23079] Generated table #253253: 482805 keys, 20499735 bytes, temperature: kWarm 2024/11/23-08:56:04.323077 139957450159680 EVENT_LOG_v1 {"time_micros": 1732323364323045, "cf_name": "default", "job": 23079, "event": "table_file_creation", "file_number": 253253, "file_size": 20499735, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3894100133, "table_properties": {"data_size": 17265360, "index_size": 919232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13518540, "raw_average_key_size": 28, "raw_value_size": 56019806, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 482805, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732323362, "oldest_key_time": 0, "file_creation_time": 1732323362, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 253253, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-08:56:04.339644 139957450159680 [compaction_job.cc:703] job-23079: subcompact[0], size: 0.608246 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-23079/att-02 [253237,253238,253239,253240,253241,253242,253243,253244,253245,253246,253247,253248,253249,253250,253251,253252,253253] 2024/11/23-08:56:04.428504 139957450159680 (Original Log Time 2024/11/23-08:55:12.189963) EVENT_LOG_v1 {"time_micros": 1732323312189936, "job": 23079, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [253171], "files_L3": [253229, 253231, 253136, 253137, 253138, 253139, 253099, 253100, 253101, 253102, 253103, 253104, 253105, 253106, 253115, 253116, 253117, 253200], "score": 0.986487, "input_data_size": 624921984, "oldest_snapshot_seqno": 3894522330} 2024/11/23-08:56:04.428508 139957450159680 (Original Log Time 2024/11/23-08:56:04.426258) [compaction_job.cc:2022] [default] [JOB 23079] Compacted 1@2 + 18@3 files to L3 => 608246297 bytes 2024/11/23-08:56:04.428509 139957450159680 (Original Log Time 2024/11/23-08:56:04.428441) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 17 0 0 0] max score 0.02, MB/sec: 12.0 rd, 11.7 wr, level 3, files in(1, 18) out(17 +0 blob) MB in(27.1, 568.9 +0.0 blob) out(580.1 +0.0 blob), read-write-amplify(43.4) write-amplify(21.4) OK, records in: 15591935, records dropped: 90124 output_compression: Snappy 2024/11/23-08:56:04.428515 139957450159680 (Original Log Time 2024/11/23-08:56:04.428485) EVENT_LOG_v1 {"time_micros": 1732323364428464, "cf": "default", "job": 23079, "event": "compaction_finished", "compaction_time_micros": 52149689, "compaction_time_cpu_micros": 22639898, "output_level": 3, "num_output_files": 17, "total_output_size": 608246297, "num_input_records": 15591935, "num_output_records": 15501811, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 410651611, "file_range_sync_nanos": 0, "file_fsync_nanos": 10982, "file_prepare_write_nanos": 42053, "lsm_state": [0, 0, 0, 17, 0, 0, 0]} 2024/11/23-08:56:04.563135 139957450159680 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732323312,"executesMs":52371,"compactionJobId":23079,"attempt":2,"compactionInputRawBytes":2229820531,"compactionInputZipBytes":624921984,"compactionOutputRawBytes":0,"compactionOutputZipBytes":608246297,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-7537baca112f2c4fcce83747cd7be56d-ad4011f8a8de936e-00"}}, response = 2024/11/23-08:56:04.563198 139957450159680 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-23079/att-02: olev 3, work 52.371 s, result 7.724 ms, install 2.274 ms, input{raw 2.077 GiB zip 595.972 MiB}