2024/11/23-19:53:54.282699 139957450159680 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00: fork to child time = 0.022020 sec 2024/11/23-19:53:54.283728 139957450159680 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00 2024/11/23-19:53:54.291986 139957450159680 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00 2024/11/23-19:53:54.293191 139957450159680 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-19:53:54.293596 139957450159680 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-19:53:54.359816 139957450159680 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34641/att-00/MANIFEST-206347 2024/11/23-19:53:54.392786 139957450159680 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34641/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 278119, last_sequence is 4228392522, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 277961 2024/11/23-19:53:54.392796 139957450159680 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-19:53:54.393149 139957450159680 [version_set.cc:5721] Creating manifest 278119 2024/11/23-19:53:54.426437 139957450159680 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-19:53:54.450424 139957450159680 [compaction_job.cc:2437] [default] [JOB 34641] Compacting 1@2 + 19@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-19:53:54.450451 139957450159680 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 25925 Base level 2, inputs: [278092(27MB)], [278052(36MB) 278053(36MB) 278054(35MB) 278055(37MB) 278056(5497KB) 278023(38MB) 278024(38MB) 278025(35MB) 278026(35MB) 278027(33MB) 278028(35MB) 278029(36MB) 278030(35MB) 278031(35MB) 278032(35MB) 278033(36MB) 278034(37MB) 278035(36MB) 278113(37MB)] 2024/11/23-19:53:54.458922 139957450159680 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:53:54.458958 139957450159680 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:53:54.458994 139957450159680 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-19:53:54.459004 139957450159680 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:53:54.459020 139957450159680 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-19:53:55.054713 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278120.sst) /dev/shm/ToplingTemp/Topling-171946-zjHx5W 2024/11/23-19:53:55.631472 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278120.sst): first pass time = 0.58's, 232.728'MB/sec 2024/11/23-19:53:57.467726 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278120.sst): old prealloc_size = 147639500, real_size = 34669536 2024/11/23-19:53:57.468323 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278120.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926858 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0327 GB avg-val = 116.81 avg-zval = 35.25 2024/11/23-19:53:57.469107 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278120.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:53:58.536967 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278120.sst): second pass time = 1.84's, 58.990'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 36.149'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3049.591'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.253'MB/sec write SST data time = 0.15's, 252.227'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 24059.775'MB/sec zip my value time = 1.84's, unzip length = 0.108'GB zip my value throughput = 58.990'MB/sec zip pipeline throughput = 108.691'MB/sec entries = 926858 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 35.25 usrkeys = 926858 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.81 avg-zval = 35.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926858 bytes = 3128160 } seq expand size = 6488006 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5145 value = 3.3134 dict = 1.00 all = 3.4725 } Zip/UnZip{ index = 0.0951 value = 0.3018 dict = 1.00 all = 0.2880 } ---------------------------- total value len = 8.963374 GB avg = 0.168 KB (by entry num) total key len = 1.466773 GB avg = 0.028 KB total ukey len = 1.016415 GB avg = 0.015 KB total ukey num = 0.066477293 Billion total entry num = 0.053285022 Billion write speed all = 0.263878500 MB/sec (with seq num) write speed all = 0.253093784 MB/sec (without seq num) 2024/11/23-19:53:58.586234 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278120: 926858 keys, 38652124 bytes, temperature: kWarm 2024/11/23-19:53:58.586320 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362838586265, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278120, "file_size": 38652124, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4136224572, "largest_seqno": 4228210088, "table_properties": {"data_size": 32674640, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952024, "raw_average_key_size": 28, "raw_value_size": 108265744, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926858, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362835, "oldest_key_time": 0, "file_creation_time": 1732362835, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278120, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:53:58.622193 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278121.sst) /dev/shm/ToplingTemp/Topling-171946-1Ja11V 2024/11/23-19:53:59.537585 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278121.sst): first pass time = 0.92's, 146.632'MB/sec 2024/11/23-19:54:01.464187 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278121.sst): old prealloc_size = 147639500, real_size = 34634800 2024/11/23-19:54:01.464850 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278121.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927052 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 116.78 avg-zval = 35.21 2024/11/23-19:54:01.465658 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278121.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:02.492631 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278121.sst): second pass time = 1.93's, 56.219'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 37.438'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2801.267'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.265'MB/sec write SST data time = 0.15's, 249.589'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 21811.466'MB/sec zip my value time = 1.93's, unzip length = 0.108'GB zip my value throughput = 56.219'MB/sec zip pipeline throughput = 101.708'MB/sec entries = 927052 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.78 avg-zval = 35.21 usrkeys = 927052 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.78 avg-zval = 35.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927052 bytes = 3012928 } seq expand size = 6489364 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5121 value = 3.3169 dict = 1.00 all = 3.4860 } Zip/UnZip{ index = 0.0951 value = 0.3015 dict = 1.00 all = 0.2869 } ---------------------------- total value len = 9.078123 GB avg = 0.167 KB (by entry num) total key len = 1.492731 GB avg = 0.028 KB total ukey len = 1.034956 GB avg = 0.015 KB total ukey num = 0.067404345 Billion total entry num = 0.054212074 Billion write speed all = 0.267411574 MB/sec (with seq num) write speed all = 0.256440323 MB/sec (without seq num) 2024/11/23-19:54:02.566200 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278121: 927052 keys, 38501922 bytes, temperature: kWarm 2024/11/23-19:54:02.566270 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362842566238, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278121, "file_size": 38501922, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4168606255, "largest_seqno": 4228210114, "table_properties": {"data_size": 32639072, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957456, "raw_average_key_size": 28, "raw_value_size": 108260376, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927052, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362838, "oldest_key_time": 0, "file_creation_time": 1732362838, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278121, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:02.607533 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278122.sst) /dev/shm/ToplingTemp/Topling-171946-5rQWNX 2024/11/23-19:54:03.871711 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278122.sst): first pass time = 1.26's, 106.174'MB/sec 2024/11/23-19:54:05.963100 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278122.sst): old prealloc_size = 147639500, real_size = 33926032 2024/11/23-19:54:05.963670 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278122.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928263 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0319 GB avg-val = 116.59 avg-zval = 34.38 2024/11/23-19:54:05.964566 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278122.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:06.992266 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278122.sst): second pass time = 2.09's, 51.770'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 36.839'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3239.269'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.23's, 166.643'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27298.247'MB/sec zip my value time = 2.09's, unzip length = 0.108'GB zip my value throughput = 51.770'MB/sec zip pipeline throughput = 132.807'MB/sec entries = 928263 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.59 avg-zval = 34.38 usrkeys = 928263 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.59 avg-zval = 34.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928263 bytes = 3132896 } seq expand size = 6497841 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4429 value = 3.3910 dict = 1.00 all = 3.5402 } Zip/UnZip{ index = 0.0958 value = 0.2949 dict = 1.00 all = 0.2825 } ---------------------------- total value len = 9.192848 GB avg = 0.167 KB (by entry num) total key len = 1.518722 GB avg = 0.028 KB total ukey len = 1.053521 GB avg = 0.015 KB total ukey num = 0.068332608 Billion total entry num = 0.055140337 Billion write speed all = 0.270940425 MB/sec (with seq num) write speed all = 0.259782586 MB/sec (without seq num) 2024/11/23-19:54:07.042465 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278122: 928263 keys, 37911992 bytes, temperature: kWarm 2024/11/23-19:54:07.042555 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362847042508, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278122, "file_size": 37911992, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4155809461, "largest_seqno": 4228210598, "table_properties": {"data_size": 31915968, "index_size": 1777792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25991364, "raw_average_key_size": 28, "raw_value_size": 108226414, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928263, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362842, "oldest_key_time": 0, "file_creation_time": 1732362842, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278122, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:07.121931 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278123.sst) /dev/shm/ToplingTemp/Topling-171946-Te8H9X 2024/11/23-19:54:08.148258 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278123.sst): first pass time = 1.03's, 130.783'MB/sec 2024/11/23-19:54:09.839995 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278123.sst): old prealloc_size = 147639500, real_size = 34613472 2024/11/23-19:54:09.840557 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278123.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932354 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0326 GB avg-val = 115.96 avg-zval = 34.97 2024/11/23-19:54:09.841323 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278123.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:10.921615 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278123.sst): second pass time = 1.69's, 64.004'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 35.590'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3333.688'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.268'MB/sec write SST data time = 0.21's, 181.005'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36026.135'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 64.004'MB/sec zip pipeline throughput = 64.004'MB/sec entries = 932354 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.96 avg-zval = 34.97 usrkeys = 932354 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.96 avg-zval = 34.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932354 bytes = 3030160 } seq expand size = 6526478 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5135 value = 3.3157 dict = 1.00 all = 3.4867 } Zip/UnZip{ index = 0.0951 value = 0.3016 dict = 1.00 all = 0.2868 } ---------------------------- total value len = 9.307486 GB avg = 0.166 KB (by entry num) total key len = 1.544828 GB avg = 0.028 KB total ukey len = 1.072169 GB avg = 0.015 KB total ukey num = 0.069264962 Billion total entry num = 0.056072691 Billion write speed all = 0.274473158 MB/sec (with seq num) write speed all = 0.263127781 MB/sec (without seq num) 2024/11/23-19:54:10.969245 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278123: 932354 keys, 38494274 bytes, temperature: kWarm 2024/11/23-19:54:10.969313 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362850969278, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278123, "file_size": 38494274, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4191256266, "largest_seqno": 4228210195, "table_properties": {"data_size": 32606544, "index_size": 1773632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26105912, "raw_average_key_size": 28, "raw_value_size": 108111920, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932354, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362847, "oldest_key_time": 0, "file_creation_time": 1732362847, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278123, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:11.005383 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278124.sst) /dev/shm/ToplingTemp/Topling-171946-NOajUY 2024/11/23-19:54:12.125412 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278124.sst): first pass time = 1.12's, 119.838'MB/sec 2024/11/23-19:54:13.752370 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278124.sst): old prealloc_size = 147639500, real_size = 36690272 2024/11/23-19:54:13.752880 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278124.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934264 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0347 GB avg-val = 115.66 avg-zval = 37.11 2024/11/23-19:54:13.753629 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278124.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:14.736946 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278124.sst): second pass time = 1.63's, 66.472'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 41.196'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3649.238'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.275'MB/sec write SST data time = 0.14's, 299.386'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38534.634'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.472'MB/sec zip pipeline throughput = 66.472'MB/sec entries = 934264 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.66 avg-zval = 37.11 usrkeys = 934264 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.39 avg-zval = 37.11 TagRS{ kinds = RS_Key0_TagN bytes = 146080 } TagArray{ size = 765226 bytes = 2869616 } seq expand size = 5356582 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0406 GB } UnZip/Zip{ index = 10.4453 value = 3.1170 dict = 1.00 all = 3.3091 } Zip/UnZip{ index = 0.0957 value = 0.3208 dict = 1.00 all = 0.3022 } ---------------------------- total value len = 9.420901 GB avg = 0.165 KB (by entry num) total key len = 1.570987 GB avg = 0.028 KB total ukey len = 1.090854 GB avg = 0.016 KB total ukey num = 0.070199226 Billion total entry num = 0.057006955 Billion write speed all = 0.277976402 MB/sec (with seq num) write speed all = 0.266443106 MB/sec (without seq num) 2024/11/23-19:54:14.791027 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278124: 934264 keys, 40560186 bytes, temperature: kWarm 2024/11/23-19:54:14.791088 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362854791056, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278124, "file_size": 40560186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4228210463, "table_properties": {"data_size": 34667664, "index_size": 1788864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26159392, "raw_average_key_size": 28, "raw_value_size": 108058340, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934264, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362850, "oldest_key_time": 0, "file_creation_time": 1732362850, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278124, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:14.833468 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278125.sst) /dev/shm/ToplingTemp/Topling-171946-zQMURX 2024/11/23-19:54:15.824452 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278125.sst): first pass time = 0.99's, 135.445'MB/sec 2024/11/23-19:54:17.417710 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278125.sst): old prealloc_size = 147639500, real_size = 35787536 2024/11/23-19:54:17.418417 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278125.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929436 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0338 GB avg-val = 116.41 avg-zval = 36.33 2024/11/23-19:54:17.419229 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278125.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:18.457098 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278125.sst): second pass time = 1.59's, 67.943'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 38.252'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2614.723'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.267'MB/sec write SST data time = 0.17's, 237.810'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31028.235'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 67.943'MB/sec zip pipeline throughput = 67.943'MB/sec entries = 929436 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.41 avg-zval = 36.33 usrkeys = 929436 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.37 avg-zval = 36.33 TagRS{ kinds = RS_Key0_TagN bytes = 145320 } TagArray{ size = 791682 bytes = 2968816 } seq expand size = 5541774 multi value expand size = 0 cnt WriteAppend = 21 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.3754 value = 3.2045 dict = 1.00 all = 3.3759 } Zip/UnZip{ index = 0.0964 value = 0.3121 dict = 1.00 all = 0.2962 } ---------------------------- total value len = 9.534636 GB avg = 0.165 KB (by entry num) total key len = 1.597011 GB avg = 0.028 KB total ukey len = 1.109443 GB avg = 0.016 KB total ukey num = 0.071128662 Billion total entry num = 0.057936391 Billion write speed all = 0.281484334 MB/sec (with seq num) write speed all = 0.269764102 MB/sec (without seq num) 2024/11/23-19:54:18.511264 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278125: 929436 keys, 39758026 bytes, temperature: kWarm 2024/11/23-19:54:18.511337 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362858511300, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278125, "file_size": 39758026, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775967, "table_properties": {"data_size": 33763392, "index_size": 1791616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26024208, "raw_average_key_size": 28, "raw_value_size": 108193648, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929436, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362854, "oldest_key_time": 0, "file_creation_time": 1732362854, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278125, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:18.547046 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278126.sst) /dev/shm/ToplingTemp/Topling-171946-V30GJW 2024/11/23-19:54:19.714746 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278126.sst): first pass time = 1.17's, 114.946'MB/sec 2024/11/23-19:54:21.158690 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278126.sst): old prealloc_size = 147639500, real_size = 32626784 2024/11/23-19:54:21.159432 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278126.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929017 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.47 avg-zval = 32.95 2024/11/23-19:54:21.160325 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278126.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:22.150734 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278126.sst): second pass time = 1.44's, 74.985'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 37.143'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2471.025'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.277'MB/sec write SST data time = 0.15's, 242.451'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33582.291'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 74.985'MB/sec zip pipeline throughput = 74.985'MB/sec entries = 929017 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.47 avg-zval = 32.95 usrkeys = 929017 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.47 avg-zval = 32.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929017 bytes = 3367696 } seq expand size = 6503119 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4465 value = 3.5343 dict = 1.00 all = 3.6425 } Zip/UnZip{ index = 0.0957 value = 0.2829 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 9.649345 GB avg = 0.164 KB (by entry num) total key len = 1.623024 GB avg = 0.028 KB total ukey len = 1.128023 GB avg = 0.016 KB total ukey num = 0.072057679 Billion total entry num = 0.058865408 Billion write speed all = 0.285016102 MB/sec (with seq num) write speed all = 0.273109047 MB/sec (without seq num) 2024/11/23-19:54:22.198208 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278126: 929017 keys, 36847354 bytes, temperature: kWarm 2024/11/23-19:54:22.198281 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362862198246, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278126, "file_size": 36847354, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3516470397, "largest_seqno": 4012776021, "table_properties": {"data_size": 30615696, "index_size": 1778624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26012476, "raw_average_key_size": 28, "raw_value_size": 108205314, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929017, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362858, "oldest_key_time": 0, "file_creation_time": 1732362858, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278126, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:22.238455 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278127.sst) /dev/shm/ToplingTemp/Topling-171946-eBqmiX 2024/11/23-19:54:23.262755 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278127.sst): first pass time = 1.02's, 131.039'MB/sec 2024/11/23-19:54:24.603712 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278127.sst): old prealloc_size = 147639500, real_size = 35411616 2024/11/23-19:54:24.604293 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278127.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923768 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0334 GB avg-val = 117.29 avg-zval = 36.17 2024/11/23-19:54:24.604969 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278127.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:25.429785 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278127.sst): second pass time = 1.34's, 80.847'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.449'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3176.886'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 302.507'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33537.189'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.847'MB/sec zip pipeline throughput = 80.847'MB/sec entries = 923768 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.29 avg-zval = 36.17 usrkeys = 923768 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.91 avg-zval = 36.17 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 740771 bytes = 2777904 } seq expand size = 5185397 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.4442 value = 3.2430 dict = 1.00 all = 3.4243 } Zip/UnZip{ index = 0.0957 value = 0.3084 dict = 1.00 all = 0.2920 } ---------------------------- total value len = 9.762882 GB avg = 0.163 KB (by entry num) total key len = 1.648889 GB avg = 0.028 KB total ukey len = 1.146498 GB avg = 0.016 KB total ukey num = 0.072981447 Billion total entry num = 0.059789176 Billion write speed all = 0.288516920 MB/sec (with seq num) write speed all = 0.276424012 MB/sec (without seq num) 2024/11/23-19:54:25.478312 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278127: 923768 keys, 39195206 bytes, temperature: kWarm 2024/11/23-19:54:25.478386 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362865478348, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278127, "file_size": 39195206, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 33411536, "index_size": 1768960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25865504, "raw_average_key_size": 28, "raw_value_size": 108352288, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923768, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362862, "oldest_key_time": 0, "file_creation_time": 1732362862, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278127, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:25.522391 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278128.sst) /dev/shm/ToplingTemp/Topling-171946-eVH1JU 2024/11/23-19:54:26.473085 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278128.sst): first pass time = 0.95's, 141.185'MB/sec 2024/11/23-19:54:27.790237 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278128.sst): old prealloc_size = 147639500, real_size = 32442720 2024/11/23-19:54:27.790770 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278128.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.79 avg-zval = 32.60 2024/11/23-19:54:27.791580 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278128.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:28.649598 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278128.sst): second pass time = 1.32's, 82.137'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.680'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3470.374'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 284.934'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30365.419'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.137'MB/sec zip pipeline throughput = 82.137'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.79 avg-zval = 32.60 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 32.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5157 value = 3.5513 dict = 1.00 all = 3.6595 } Zip/UnZip{ index = 0.0951 value = 0.2816 dict = 1.00 all = 0.2733 } ---------------------------- total value len = 9.877498 GB avg = 0.163 KB (by entry num) total key len = 1.675026 GB avg = 0.028 KB total ukey len = 1.165167 GB avg = 0.016 KB total ukey num = 0.073914904 Billion total entry num = 0.060722633 Billion write speed all = 0.292051695 MB/sec (with seq num) write speed all = 0.279770986 MB/sec (without seq num) 2024/11/23-19:54:28.695665 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278128: 933457 keys, 36676934 bytes, temperature: kWarm 2024/11/23-19:54:28.695739 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362868695703, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278128, "file_size": 36676934, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 30433808, "index_size": 1775360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362865, "oldest_key_time": 0, "file_creation_time": 1732362865, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278128, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:28.732566 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278129.sst) /dev/shm/ToplingTemp/Topling-171946-XDgziZ 2024/11/23-19:54:29.565854 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278129.sst): first pass time = 0.83's, 161.079'MB/sec 2024/11/23-19:54:30.880704 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278129.sst): old prealloc_size = 147639500, real_size = 32664448 2024/11/23-19:54:30.881290 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278129.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934399 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.64 avg-zval = 32.79 2024/11/23-19:54:30.882143 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278129.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:31.737136 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278129.sst): second pass time = 1.31's, 82.270'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.089'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3172.801'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 289.636'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32560.244'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.270'MB/sec zip pipeline throughput = 82.270'MB/sec entries = 934399 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.64 avg-zval = 32.79 usrkeys = 934399 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.64 avg-zval = 32.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934399 bytes = 3387216 } seq expand size = 6540793 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4450 value = 3.5264 dict = 1.00 all = 3.6371 } Zip/UnZip{ index = 0.0957 value = 0.2836 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 9.992093 GB avg = 0.162 KB (by entry num) total key len = 1.701189 GB avg = 0.028 KB total ukey len = 1.183855 GB avg = 0.016 KB total ukey num = 0.074849303 Billion total entry num = 0.061657032 Billion write speed all = 0.295587048 MB/sec (with seq num) write speed all = 0.283118337 MB/sec (without seq num) 2024/11/23-19:54:31.783720 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278129: 934399 keys, 36901988 bytes, temperature: kWarm 2024/11/23-19:54:31.783793 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362871783756, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278129, "file_size": 36901988, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30641456, "index_size": 1789184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26163172, "raw_average_key_size": 28, "raw_value_size": 108054646, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934399, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362868, "oldest_key_time": 0, "file_creation_time": 1732362868, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278129, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:31.820479 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278130.sst) /dev/shm/ToplingTemp/Topling-171946-wbGreX 2024/11/23-19:54:32.852491 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278130.sst): first pass time = 1.03's, 130.059'MB/sec 2024/11/23-19:54:34.205781 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278130.sst): old prealloc_size = 147639500, real_size = 35046080 2024/11/23-19:54:34.206365 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278130.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925568 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0330 GB avg-val = 117.01 avg-zval = 35.70 2024/11/23-19:54:34.207188 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278130.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:35.045338 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278130.sst): second pass time = 1.35's, 80.080'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.897'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3141.616'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 314.677'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33408.101'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.080'MB/sec zip pipeline throughput = 80.080'MB/sec entries = 925568 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.01 avg-zval = 35.70 usrkeys = 925568 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.01 avg-zval = 35.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925568 bytes = 3470896 } seq expand size = 6478976 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4434 value = 3.2777 dict = 1.00 all = 3.4090 } Zip/UnZip{ index = 0.0958 value = 0.3051 dict = 1.00 all = 0.2933 } ---------------------------- total value len = 10.106874 GB avg = 0.161 KB (by entry num) total key len = 1.727105 GB avg = 0.028 KB total ukey len = 1.202367 GB avg = 0.016 KB total ukey num = 0.075774871 Billion total entry num = 0.062582600 Billion write speed all = 0.299118616 MB/sec (with seq num) write speed all = 0.286463789 MB/sec (without seq num) 2024/11/23-19:54:35.092784 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278130: 925568 keys, 39371826 bytes, temperature: kWarm 2024/11/23-19:54:35.092856 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362875092819, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278130, "file_size": 39371826, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 33041968, "index_size": 1772544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25915904, "raw_average_key_size": 28, "raw_value_size": 108301824, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925568, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362871, "oldest_key_time": 0, "file_creation_time": 1732362871, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278130, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:35.146410 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278131.sst) /dev/shm/ToplingTemp/Topling-171946-do372V 2024/11/23-19:54:35.985157 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278131.sst): first pass time = 0.84's, 160.028'MB/sec 2024/11/23-19:54:37.270883 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278131.sst): old prealloc_size = 147639500, real_size = 32388624 2024/11/23-19:54:37.271452 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278131.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931242 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.13 avg-zval = 32.63 2024/11/23-19:54:37.272274 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278131.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:38.118764 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278131.sst): second pass time = 1.28's, 84.159'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.186'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3238.838'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 290.829'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32682.507'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.159'MB/sec zip pipeline throughput = 84.159'MB/sec entries = 931242 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.13 avg-zval = 32.63 usrkeys = 931242 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.13 avg-zval = 32.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931242 bytes = 3375760 } seq expand size = 6518694 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5108 value = 3.5592 dict = 1.00 all = 3.6656 } Zip/UnZip{ index = 0.0951 value = 0.2810 dict = 1.00 all = 0.2728 } ---------------------------- total value len = 10.221536 GB avg = 0.161 KB (by entry num) total key len = 1.753180 GB avg = 0.028 KB total ukey len = 1.220992 GB avg = 0.016 KB total ukey num = 0.076706113 Billion total entry num = 0.063513842 Billion write speed all = 0.302652394 MB/sec (with seq num) write speed all = 0.289810258 MB/sec (without seq num) 2024/11/23-19:54:38.164114 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278131: 931242 keys, 36615816 bytes, temperature: kWarm 2024/11/23-19:54:38.164186 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362878164149, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278131, "file_size": 36615816, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776300, "table_properties": {"data_size": 30383680, "index_size": 1771968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26074776, "raw_average_key_size": 28, "raw_value_size": 108142972, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362875, "oldest_key_time": 0, "file_creation_time": 1732362875, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278131, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:38.201572 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278132.sst) /dev/shm/ToplingTemp/Topling-171946-srfiaX 2024/11/23-19:54:39.151197 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278132.sst): first pass time = 0.95's, 141.343'MB/sec 2024/11/23-19:54:40.482934 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278132.sst): old prealloc_size = 147639500, real_size = 34980288 2024/11/23-19:54:40.483504 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278132.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922697 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0330 GB avg-val = 117.46 avg-zval = 35.76 2024/11/23-19:54:40.484344 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278132.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:41.327341 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278132.sst): second pass time = 1.33's, 81.433'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.541'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3206.536'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.323'MB/sec write SST data time = 0.13's, 304.945'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 24440.768'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.433'MB/sec zip pipeline throughput = 81.433'MB/sec entries = 922697 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.46 avg-zval = 35.76 usrkeys = 922697 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.46 avg-zval = 35.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922697 bytes = 3460128 } seq expand size = 6458879 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.5146 value = 3.2849 dict = 1.00 all = 3.4153 } Zip/UnZip{ index = 0.0951 value = 0.3044 dict = 1.00 all = 0.2928 } ---------------------------- total value len = 10.336377 GB avg = 0.160 KB (by entry num) total key len = 1.779016 GB avg = 0.028 KB total ukey len = 1.239446 GB avg = 0.016 KB total ukey num = 0.077628810 Billion total entry num = 0.064436539 Billion write speed all = 0.306183067 MB/sec (with seq num) write speed all = 0.293155423 MB/sec (without seq num) 2024/11/23-19:54:41.374183 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278132: 922697 keys, 39298514 bytes, temperature: kWarm 2024/11/23-19:54:41.374263 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362881374227, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278132, "file_size": 39298514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 32994352, "index_size": 1755072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835516, "raw_average_key_size": 28, "raw_value_size": 108382282, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922697, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362878, "oldest_key_time": 0, "file_creation_time": 1732362878, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278132, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:41.410099 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278133.sst) /dev/shm/ToplingTemp/Topling-171946-1Sc9GW 2024/11/23-19:54:42.324527 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278133.sst): first pass time = 0.91's, 146.784'MB/sec 2024/11/23-19:54:43.637506 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278133.sst): old prealloc_size = 147639500, real_size = 32815824 2024/11/23-19:54:43.638095 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278133.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.15 avg-zval = 33.07 2024/11/23-19:54:43.638928 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278133.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:44.497246 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278133.sst): second pass time = 1.31's, 82.420'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.223'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3159.687'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 295.098'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33333.457'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.420'MB/sec zip pipeline throughput = 82.420'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.15 avg-zval = 33.07 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.15 avg-zval = 33.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.3752 value = 3.5126 dict = 1.00 all = 3.6120 } Zip/UnZip{ index = 0.0964 value = 0.2847 dict = 1.00 all = 0.2769 } ---------------------------- total value len = 10.451042 GB avg = 0.160 KB (by entry num) total key len = 1.805086 GB avg = 0.028 KB total ukey len = 1.258067 GB avg = 0.016 KB total ukey num = 0.078559893 Billion total entry num = 0.065367622 Billion write speed all = 0.309714955 MB/sec (with seq num) write speed all = 0.296500125 MB/sec (without seq num) 2024/11/23-19:54:44.543701 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278133: 931083 keys, 37158896 bytes, temperature: kWarm 2024/11/23-19:54:44.543771 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362884543733, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278133, "file_size": 37158896, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 30788032, "index_size": 1794816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362881, "oldest_key_time": 0, "file_creation_time": 1732362881, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278133, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:44.581514 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278134.sst) /dev/shm/ToplingTemp/Topling-171946-LmYdAV 2024/11/23-19:54:45.607882 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278134.sst): first pass time = 1.03's, 130.774'MB/sec 2024/11/23-19:54:46.990043 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278134.sst): old prealloc_size = 147639500, real_size = 34502928 2024/11/23-19:54:46.990624 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278134.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928181 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.60 avg-zval = 35.01 2024/11/23-19:54:46.991144 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278134.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:47.783622 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278134.sst): second pass time = 1.38's, 78.394'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 47.109'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3180.204'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.348'MB/sec write SST data time = 0.13's, 294.880'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34177.208'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.394'MB/sec zip pipeline throughput = 78.394'MB/sec entries = 928181 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.60 avg-zval = 35.01 usrkeys = 928181 avg-key = 20.00 avg-zkey = 1.92 avg-val = 120.65 avg-zval = 35.01 TagRS{ kinds = RS_Key0_TagN bytes = 145120 } TagArray{ size = 536440 bytes = 1877552 } seq expand size = 3755080 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4427 value = 3.3308 dict = 1.00 all = 3.5903 } Zip/UnZip{ index = 0.0958 value = 0.3002 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 10.563026 GB avg = 0.159 KB (by entry num) total key len = 1.831075 GB avg = 0.028 KB total ukey len = 1.276631 GB avg = 0.016 KB total ukey num = 0.079488074 Billion total entry num = 0.066295803 Billion write speed all = 0.313175550 MB/sec (with seq num) write speed all = 0.299774190 MB/sec (without seq num) 2024/11/23-19:54:47.830028 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278134: 928181 keys, 37383801 bytes, temperature: kWarm 2024/11/23-19:54:47.830103 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362887830065, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278134, "file_size": 37383801, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 32493056, "index_size": 1777664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989068, "raw_average_key_size": 28, "raw_value_size": 108228706, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928181, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362884, "oldest_key_time": 0, "file_creation_time": 1732362884, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278134, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:47.877190 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278135.sst) /dev/shm/ToplingTemp/Topling-171946-Re9PjW 2024/11/23-19:54:48.901266 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278135.sst): first pass time = 1.02's, 131.069'MB/sec 2024/11/23-19:54:50.392552 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278135.sst): old prealloc_size = 147639500, real_size = 36891088 2024/11/23-19:54:50.393106 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278135.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926855 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.81 avg-zval = 37.64 2024/11/23-19:54:50.393357 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278135.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:51.112837 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278135.sst): second pass time = 1.49's, 72.638'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 53.281'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3338.931'MB/sec (index lex order gen) rebuild zvType time = 0.60's, 0.389'MB/sec write SST data time = 0.12's, 309.209'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26640.823'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.638'MB/sec zip pipeline throughput = 72.638'MB/sec entries = 926855 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.81 avg-zval = 37.64 usrkeys = 926855 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.98 avg-zval = 37.64 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 154398 bytes = 482512 } seq expand size = 1080786 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.4458 value = 3.1035 dict = 1.00 all = 3.4974 } Zip/UnZip{ index = 0.0957 value = 0.3222 dict = 1.00 all = 0.2859 } ---------------------------- total value len = 10.672372 GB avg = 0.159 KB (by entry num) total key len = 1.857027 GB avg = 0.028 KB total ukey len = 1.295168 GB avg = 0.016 KB total ukey num = 0.080414929 Billion total entry num = 0.067222658 Billion write speed all = 0.316567658 MB/sec (with seq num) write speed all = 0.302980083 MB/sec (without seq num) 2024/11/23-19:54:51.160551 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278135: 926855 keys, 38376647 bytes, temperature: kWarm 2024/11/23-19:54:51.160622 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362891160585, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278135, "file_size": 38376647, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700812, "table_properties": {"data_size": 34884608, "index_size": 1774592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951940, "raw_average_key_size": 28, "raw_value_size": 108265806, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926855, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362887, "oldest_key_time": 0, "file_creation_time": 1732362887, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278135, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:51.199651 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278136.sst) /dev/shm/ToplingTemp/Topling-171946-KtdxTX 2024/11/23-19:54:52.239317 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278136.sst): first pass time = 1.04's, 129.102'MB/sec 2024/11/23-19:54:53.569960 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278136.sst): old prealloc_size = 147639500, real_size = 34353168 2024/11/23-19:54:53.570504 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278136.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924401 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 117.19 avg-zval = 35.01 2024/11/23-19:54:53.571199 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278136.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:54.396905 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278136.sst): second pass time = 1.33's, 81.502'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 46.089'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3497.806'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 297.783'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33849.909'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.502'MB/sec zip pipeline throughput = 81.502'MB/sec entries = 924401 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.19 avg-zval = 35.01 usrkeys = 924401 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 35.01 TagRS{ kinds = RS_Key0_TagN bytes = 144520 } TagArray{ size = 787006 bytes = 2754528 } seq expand size = 5509042 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5130 value = 3.3475 dict = 1.00 all = 3.5216 } Zip/UnZip{ index = 0.0951 value = 0.2987 dict = 1.00 all = 0.2840 } ---------------------------- total value len = 10.786216 GB avg = 0.158 KB (by entry num) total key len = 1.882910 GB avg = 0.028 KB total ukey len = 1.313656 GB avg = 0.016 KB total ukey num = 0.081339330 Billion total entry num = 0.068147059 Billion write speed all = 0.320071435 MB/sec (with seq num) write speed all = 0.306298155 MB/sec (without seq num) 2024/11/23-19:54:54.449488 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278136: 924401 keys, 38112248 bytes, temperature: kWarm 2024/11/23-19:54:54.449558 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362894449521, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278136, "file_size": 38112248, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4228210546, "table_properties": {"data_size": 32363264, "index_size": 1758592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25883228, "raw_average_key_size": 28, "raw_value_size": 108334510, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924401, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362891, "oldest_key_time": 0, "file_creation_time": 1732362891, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278136, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:54.492801 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278137.sst) /dev/shm/ToplingTemp/Topling-171946-jJHuVW 2024/11/23-19:54:55.548302 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278137.sst): first pass time = 1.06's, 127.165'MB/sec 2024/11/23-19:54:56.912684 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278137.sst): old prealloc_size = 147639500, real_size = 35244128 2024/11/23-19:54:56.913295 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278137.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931040 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0332 GB avg-val = 116.16 avg-zval = 35.69 2024/11/23-19:54:56.914043 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278137.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-19:54:57.746871 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278137.sst): second pass time = 1.36's, 79.308'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.039'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3022.190'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 309.032'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30717.716'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.308'MB/sec zip pipeline throughput = 79.308'MB/sec entries = 931040 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.16 avg-zval = 35.69 usrkeys = 931040 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.16 avg-zval = 35.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931040 bytes = 3142272 } seq expand size = 6517280 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.4470 value = 3.2547 dict = 1.00 all = 3.4206 } Zip/UnZip{ index = 0.0957 value = 0.3073 dict = 1.00 all = 0.2923 } ---------------------------- total value len = 10.900882 GB avg = 0.158 KB (by entry num) total key len = 1.908979 GB avg = 0.028 KB total ukey len = 1.332277 GB avg = 0.016 KB total ukey num = 0.082270370 Billion total entry num = 0.069078099 Billion write speed all = 0.323599562 MB/sec (with seq num) write speed all = 0.309639290 MB/sec (without seq num) 2024/11/23-19:54:57.800377 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278137: 931040 keys, 39237788 bytes, temperature: kWarm 2024/11/23-19:54:57.800447 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362897800411, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278137, "file_size": 39237788, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4228210497, "table_properties": {"data_size": 33228816, "index_size": 1782400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26069120, "raw_average_key_size": 28, "raw_value_size": 108148608, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931040, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362894, "oldest_key_time": 0, "file_creation_time": 1732362894, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278137, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:57.835687 139957450159680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278138.sst) /dev/shm/ToplingTemp/Topling-171946-OVh65V 2024/11/23-19:54:57.987422 139957450159680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278138.sst): first pass time = 0.15's, 242.623'MB/sec 2024/11/23-19:54:58.325357 139957450159680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278138.sst): old prealloc_size = 147639500, real_size = 10184432 2024/11/23-19:54:58.325560 139957450159680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278138.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 256004 prefix = 4 raw-key = 0.0041 GB zip-key = 0.0005 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0296 GB zip-val = 0.0096 GB avg-val = 115.77 avg-zval = 37.63 2024/11/23-19:54:58.325788 139957450159680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278138.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0006 GB, waited 0.000 sec, Key+Value = 0.037 GB 2024/11/23-19:54:58.552965 139957450159680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00/278138.sst): second pass time = 0.34's, 87.888'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.23's, 49.325'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2713.351'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.336'MB/sec write SST data time = 0.04's, 304.179'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.302'MB, throughput = 10453.430'MB/sec zip my value time = 0.34's, unzip length = 0.030'GB zip my value throughput = 87.888'MB/sec zip pipeline throughput = 87.888'MB/sec entries = 256004 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.77 avg-zval = 37.63 usrkeys = 256004 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.77 avg-zval = 37.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 256004 bytes = 800032 } seq expand size = 1792028 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0051 GB value = 0.0296 GB dict = 0.30 MB all = 0.0368 GB } __ZipSize{ index = 0.0005 GB value = 0.0096 GB dict = 0.30 MB all = 0.0112 GB } UnZip/Zip{ index = 10.4906 value = 3.0770 dict = 1.00 all = 3.2788 } Zip/UnZip{ index = 0.0953 value = 0.3250 dict = 1.00 all = 0.3050 } ---------------------------- total value len = 10.932311 GB avg = 0.158 KB (by entry num) total key len = 1.916147 GB avg = 0.028 KB total ukey len = 1.337397 GB avg = 0.016 KB total ukey num = 0.082526374 Billion total entry num = 0.069334103 Billion write speed all = 0.324568004 MB/sec (with seq num) write speed all = 0.310556281 MB/sec (without seq num) 2024/11/23-19:54:58.574574 139957450159680 [compaction_job.cc:1948] [default] [JOB 34641] Generated table #278138: 256004 keys, 11225429 bytes, temperature: kWarm 2024/11/23-19:54:58.574640 139957450159680 EVENT_LOG_v1 {"time_micros": 1732362898574607, "cf_name": "default", "job": 34641, "event": "table_file_creation", "file_number": 278138, "file_size": 11225429, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4205614773, "largest_seqno": 4228209829, "table_properties": {"data_size": 9632160, "index_size": 488064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 7168112, "raw_average_key_size": 28, "raw_value_size": 29637788, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 256004, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732362897, "oldest_key_time": 0, "file_creation_time": 1732362897, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278138, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-19:54:58.589179 139957450159680 [compaction_job.cc:703] job-34641: subcompact[0], size: 0.700281 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34641/att-00 [278120,278121,278122,278123,278124,278125,278126,278127,278128,278129,278130,278131,278132,278133,278134,278135,278136,278137,278138] 2024/11/23-19:54:58.702117 139957450159680 (Original Log Time 2024/11/23-19:53:54.450508) EVENT_LOG_v1 {"time_micros": 1732362834450465, "job": 34641, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [278092], "files_L3": [278052, 278053, 278054, 278055, 278056, 278023, 278024, 278025, 278026, 278027, 278028, 278029, 278030, 278031, 278032, 278033, 278034, 278035, 278113], "score": 0.995816, "input_data_size": 720034587, "oldest_snapshot_seqno": 4228392400} 2024/11/23-19:54:58.702121 139957450159680 (Original Log Time 2024/11/23-19:54:58.700546) [compaction_job.cc:2022] [default] [JOB 34641] Compacted 1@2 + 19@3 files to L3 => 700280971 bytes 2024/11/23-19:54:58.702122 139957450159680 (Original Log Time 2024/11/23-19:54:58.702046) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 19 0 0 0] max score 0.03, MB/sec: 11.2 rd, 10.9 wr, level 3, files in(1, 19) out(19 +0 blob) MB in(27.1, 659.6 +0.0 blob) out(667.8 +0.0 blob), read-write-amplify(50.0) write-amplify(24.6) OK, records in: 17071391, records dropped: 95452 output_compression: Snappy 2024/11/23-19:54:58.702126 139957450159680 (Original Log Time 2024/11/23-19:54:58.702092) EVENT_LOG_v1 {"time_micros": 1732362898702070, "cf": "default", "job": 34641, "event": "compaction_finished", "compaction_time_micros": 64138679, "compaction_time_cpu_micros": 26924718, "output_level": 3, "num_output_files": 19, "total_output_size": 700280971, "num_input_records": 17071391, "num_output_records": 16975939, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 524787032, "file_range_sync_nanos": 0, "file_fsync_nanos": 13773, "file_prepare_write_nanos": 58287, "lsm_state": [0, 0, 0, 19, 0, 0, 0]} 2024/11/23-19:54:58.837072 139957450159680 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732362834,"executesMs":64411,"compactionJobId":34641,"attempt":0,"compactionInputRawBytes":2466280234,"compactionInputZipBytes":720034587,"compactionOutputRawBytes":0,"compactionOutputZipBytes":700280971,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c689e1f9c5ee47bc23e246151531e114-fa971a3b9d1d084b-00"}}, response = 2024/11/23-19:54:58.837176 139957450159680 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34641/att-00: olev 3, work 64.412 s, result 6.046 ms, install 1.595 ms, input{raw 2.297 GiB zip 686.678 MiB}