2024/11/22-23:07:35.116891 140024709862976 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00: fork to child time = 0.009417 sec 2024/11/22-23:07:35.117729 140024709862976 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00 2024/11/22-23:07:35.125632 140024709862976 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00 2024/11/22-23:07:35.126766 140024709862976 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-23:07:35.127034 140024709862976 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-23:07:35.173073 140024709862976 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13556/att-00/MANIFEST-206347 2024/11/22-23:07:35.186304 140024709862976 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13556/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 232966, last_sequence is 3631612171, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 232828 2024/11/22-23:07:35.186312 140024709862976 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-23:07:35.186422 140024709862976 [version_set.cc:5721] Creating manifest 232966 2024/11/22-23:07:35.216639 140024709862976 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-23:07:35.243335 140024709862976 [compaction_job.cc:2437] [default] [JOB 13556] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-23:07:35.243356 140024709862976 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 10179 Base level 2, inputs: [232963(27MB)], [232911(34MB) 232912(35MB) 232913(35MB) 232914(35MB) 232915(35MB) 232916(33MB) 232917(35MB) 232918(34MB) 232919(35MB) 232920(37MB) 232921(35MB) 232922(36MB) 232923(35MB) 232924(35MB) 232927(37MB)] 2024/11/22-23:07:35.249685 140024709862976 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:07:35.249718 140024709862976 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:07:35.249740 140024709862976 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-23:07:35.249745 140024709862976 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:07:35.249756 140024709862976 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-23:07:35.711931 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232967.sst) /dev/shm/ToplingTemp/Topling-326652-eZlvA4 2024/11/22-23:07:36.317922 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232967.sst): first pass time = 0.61's, 221.500'MB/sec 2024/11/22-23:07:38.079551 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232967.sst): old prealloc_size = 147639500, real_size = 32544880 2024/11/22-23:07:38.080156 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232967.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925015 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 117.10 avg-zval = 33.03 2024/11/22-23:07:38.080964 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232967.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:07:39.512139 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232967.sst): second pass time = 1.76's, 61.529'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.43's, 25.494'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2998.894'MB/sec (index lex order gen) rebuild zvType time = 1.30's, 0.178'MB/sec write SST data time = 0.13's, 277.970'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33890.213'MB/sec zip my value time = 1.76's, unzip length = 0.108'GB zip my value throughput = 61.529'MB/sec zip pipeline throughput = 70.988'MB/sec entries = 925015 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 33.03 usrkeys = 925015 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 33.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925015 bytes = 3121936 } seq expand size = 6475105 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5138 value = 3.5451 dict = 1.00 all = 3.6749 } Zip/UnZip{ index = 0.0951 value = 0.2821 dict = 1.00 all = 0.2721 } ---------------------------- total value len = 0.114792 GB avg = 0.124 KB (by entry num) total key len = 0.025900 GB avg = 0.028 KB total ukey len = 0.018500 GB avg = 0.020 KB total ukey num = 0.000925015 Billion total entry num = 0.000925015 Billion write speed all = 0.002741673 MB/sec (with seq num) write speed all = 0.002597468 MB/sec (without seq num) 2024/11/22-23:07:39.548340 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232967: 925015 keys, 36522454 bytes, temperature: kWarm 2024/11/22-23:07:39.548423 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288059548371, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232967, "file_size": 36522454, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3513012344, "largest_seqno": 3631594408, "table_properties": {"data_size": 30553824, "index_size": 1759616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900420, "raw_average_key_size": 28, "raw_value_size": 108317330, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925015, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288055, "oldest_key_time": 0, "file_creation_time": 1732288055, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232967, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:07:39.591605 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232968.sst) /dev/shm/ToplingTemp/Topling-326652-TiLAa7 2024/11/22-23:07:40.574824 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232968.sst): first pass time = 0.98's, 136.514'MB/sec 2024/11/22-23:07:44.846386 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232968.sst): old prealloc_size = 147639500, real_size = 33395952 2024/11/22-23:07:44.846950 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232968.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924806 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.13 avg-zval = 33.96 2024/11/22-23:07:44.847733 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232968.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:07:46.176223 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232968.sst): second pass time = 4.27's, 25.363'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.33's, 28.191'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3334.016'MB/sec (index lex order gen) rebuild zvType time = 1.10's, 0.211'MB/sec write SST data time = 0.23's, 161.812'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31952.899'MB/sec zip my value time = 4.27's, unzip length = 0.108'GB zip my value throughput = 25.363'MB/sec zip pipeline throughput = 134.001'MB/sec entries = 924806 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.13 avg-zval = 33.96 usrkeys = 924806 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.13 avg-zval = 33.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924806 bytes = 3236832 } seq expand size = 6473642 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5130 value = 3.4492 dict = 1.00 all = 3.5802 } Zip/UnZip{ index = 0.0951 value = 0.2899 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 0.229589 GB avg = 0.124 KB (by entry num) total key len = 0.051795 GB avg = 0.028 KB total ukey len = 0.036996 GB avg = 0.020 KB total ukey num = 0.001849821 Billion total entry num = 0.001849821 Billion write speed all = 0.005482607 MB/sec (with seq num) write speed all = 0.005194265 MB/sec (without seq num) 2024/11/22-23:07:46.220835 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232968: 924806 keys, 37488790 bytes, temperature: kWarm 2024/11/22-23:07:46.220912 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288066220868, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232968, "file_size": 37488790, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3631594030, "table_properties": {"data_size": 31405216, "index_size": 1759360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25894568, "raw_average_key_size": 28, "raw_value_size": 108323200, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924806, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288059, "oldest_key_time": 0, "file_creation_time": 1732288059, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232968, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:07:46.288611 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232969.sst) /dev/shm/ToplingTemp/Topling-326652-nug4B5 2024/11/22-23:07:48.034444 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232969.sst): first pass time = 1.75's, 76.881'MB/sec 2024/11/22-23:07:50.024372 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232969.sst): old prealloc_size = 147639500, real_size = 32996160 2024/11/22-23:07:50.024911 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232969.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925108 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.08 avg-zval = 33.51 2024/11/22-23:07:50.025668 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232969.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:07:51.463005 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232969.sst): second pass time = 1.99's, 54.449'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.44's, 25.782'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3398.195'MB/sec (index lex order gen) rebuild zvType time = 1.20's, 0.192'MB/sec write SST data time = 0.23's, 158.727'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31992.305'MB/sec zip my value time = 1.99's, unzip length = 0.108'GB zip my value throughput = 54.449'MB/sec zip pipeline throughput = 125.962'MB/sec entries = 925108 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 33.51 usrkeys = 925108 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 33.51 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925108 bytes = 3237888 } seq expand size = 6475756 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5126 value = 3.4935 dict = 1.00 all = 3.6187 } Zip/UnZip{ index = 0.0951 value = 0.2862 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 0.344380 GB avg = 0.124 KB (by entry num) total key len = 0.077698 GB avg = 0.028 KB total ukey len = 0.055499 GB avg = 0.020 KB total ukey num = 0.002774929 Billion total entry num = 0.002774929 Billion write speed all = 0.008223090 MB/sec (with seq num) write speed all = 0.007790592 MB/sec (without seq num) 2024/11/22-23:07:51.499946 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232969: 925108 keys, 37089548 bytes, temperature: kWarm 2024/11/22-23:07:51.500027 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288071499984, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232969, "file_size": 37089548, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3494389278, "largest_seqno": 3631592210, "table_properties": {"data_size": 31004720, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903024, "raw_average_key_size": 28, "raw_value_size": 108314748, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925108, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288066, "oldest_key_time": 0, "file_creation_time": 1732288066, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232969, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:07:51.569393 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232970.sst) /dev/shm/ToplingTemp/Topling-326652-xroLz4 2024/11/22-23:07:52.525953 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232970.sst): first pass time = 0.96's, 140.318'MB/sec 2024/11/22-23:07:54.707772 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232970.sst): old prealloc_size = 147639500, real_size = 33588240 2024/11/22-23:07:54.708312 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232970.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924899 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.12 avg-zval = 34.16 2024/11/22-23:07:54.709078 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232970.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:07:55.656101 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232970.sst): second pass time = 2.18's, 49.663'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.735'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3440.683'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 288.351'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27623.853'MB/sec zip my value time = 2.18's, unzip length = 0.108'GB zip my value throughput = 49.663'MB/sec zip pipeline throughput = 156.132'MB/sec entries = 924899 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.12 avg-zval = 34.16 usrkeys = 924899 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.12 avg-zval = 34.16 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924899 bytes = 3237168 } seq expand size = 6474293 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5175 value = 3.4281 dict = 1.00 all = 3.5619 } Zip/UnZip{ index = 0.0951 value = 0.2917 dict = 1.00 all = 0.2807 } ---------------------------- total value len = 0.459175 GB avg = 0.124 KB (by entry num) total key len = 0.103595 GB avg = 0.028 KB total ukey len = 0.073997 GB avg = 0.020 KB total ukey num = 0.003699828 Billion total entry num = 0.003699828 Billion write speed all = 0.010963216 MB/sec (with seq num) write speed all = 0.010386611 MB/sec (without seq num) 2024/11/22-23:07:55.708082 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232970: 924899 keys, 37681122 bytes, temperature: kWarm 2024/11/22-23:07:55.708140 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288075708105, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232970, "file_size": 37681122, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3631593325, "table_properties": {"data_size": 31598016, "index_size": 1758784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897172, "raw_average_key_size": 28, "raw_value_size": 108320690, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924899, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288071, "oldest_key_time": 0, "file_creation_time": 1732288071, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232970, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:07:55.751643 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232971.sst) /dev/shm/ToplingTemp/Topling-326652-VksYr4 2024/11/22-23:07:56.551170 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232971.sst): first pass time = 0.80's, 167.879'MB/sec 2024/11/22-23:07:58.303726 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232971.sst): old prealloc_size = 147639500, real_size = 32926048 2024/11/22-23:07:58.304422 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232971.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927842 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.66 avg-zval = 33.33 2024/11/22-23:07:58.305244 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232971.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:07:59.434321 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232971.sst): second pass time = 1.75's, 61.786'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.13's, 32.648'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2609.932'MB/sec (index lex order gen) rebuild zvType time = 0.97's, 0.240'MB/sec write SST data time = 0.16's, 226.889'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32183.704'MB/sec zip my value time = 1.75's, unzip length = 0.108'GB zip my value throughput = 61.786'MB/sec zip pipeline throughput = 123.612'MB/sec entries = 927842 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 33.33 usrkeys = 927842 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 33.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927842 bytes = 3131488 } seq expand size = 6494894 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5116 value = 3.4996 dict = 1.00 all = 3.6362 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2750 } ---------------------------- total value len = 0.573908 GB avg = 0.124 KB (by entry num) total key len = 0.129575 GB avg = 0.028 KB total ukey len = 0.092553 GB avg = 0.020 KB total ukey num = 0.004627670 Billion total entry num = 0.004627670 Billion write speed all = 0.013703406 MB/sec (with seq num) write speed all = 0.012982253 MB/sec (without seq num) 2024/11/22-23:07:59.468148 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232971: 927842 keys, 36911268 bytes, temperature: kWarm 2024/11/22-23:07:59.468208 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288079468177, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232971, "file_size": 36911268, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3523718385, "largest_seqno": 3631592928, "table_properties": {"data_size": 30928528, "index_size": 1765376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979576, "raw_average_key_size": 28, "raw_value_size": 108238252, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927842, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288075, "oldest_key_time": 0, "file_creation_time": 1732288075, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232971, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:07:59.504613 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232972.sst) /dev/shm/ToplingTemp/Topling-326652-NwO0Y6 2024/11/22-23:08:00.630098 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232972.sst): first pass time = 1.13's, 119.257'MB/sec 2024/11/22-23:08:02.627102 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232972.sst): old prealloc_size = 147639500, real_size = 32082064 2024/11/22-23:08:02.627684 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232972.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924638 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0301 GB avg-val = 117.16 avg-zval = 32.53 2024/11/22-23:08:02.628476 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232972.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:08:03.511528 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232972.sst): second pass time = 2.00's, 54.263'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.902'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3166.931'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.306'MB/sec write SST data time = 0.13's, 281.068'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33308.278'MB/sec zip my value time = 2.00's, unzip length = 0.108'GB zip my value throughput = 54.263'MB/sec zip pipeline throughput = 148.705'MB/sec entries = 924638 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.16 avg-zval = 32.53 usrkeys = 924638 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.16 avg-zval = 32.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924638 bytes = 3236240 } seq expand size = 6472466 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4438 value = 3.6013 dict = 1.00 all = 3.7103 } Zip/UnZip{ index = 0.0958 value = 0.2777 dict = 1.00 all = 0.2695 } ---------------------------- total value len = 0.688708 GB avg = 0.124 KB (by entry num) total key len = 0.155465 GB avg = 0.028 KB total ukey len = 0.111046 GB avg = 0.020 KB total ukey num = 0.005552308 Billion total entry num = 0.005552308 Billion write speed all = 0.016442658 MB/sec (with seq num) write speed all = 0.015577483 MB/sec (without seq num) 2024/11/22-23:08:03.545535 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232972: 924638 keys, 36174228 bytes, temperature: kWarm 2024/11/22-23:08:03.545592 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288083545562, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232972, "file_size": 36174228, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3631594621, "table_properties": {"data_size": 30080064, "index_size": 1770688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25889864, "raw_average_key_size": 28, "raw_value_size": 108327892, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924638, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288079, "oldest_key_time": 0, "file_creation_time": 1732288079, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232972, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:08:03.584480 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232973.sst) /dev/shm/ToplingTemp/Topling-326652-Dh8Gw4 2024/11/22-23:08:04.483994 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232973.sst): first pass time = 0.90's, 149.218'MB/sec 2024/11/22-23:08:06.174953 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232973.sst): old prealloc_size = 147639500, real_size = 32636752 2024/11/22-23:08:06.175494 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232973.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927883 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.65 avg-zval = 33.02 2024/11/22-23:08:06.176274 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232973.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:08:07.171485 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232973.sst): second pass time = 1.69's, 64.033'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 36.867'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3476.260'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.304'MB/sec write SST data time = 0.23's, 157.724'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34739.633'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 64.033'MB/sec zip pipeline throughput = 96.508'MB/sec entries = 927883 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.65 avg-zval = 33.02 usrkeys = 927883 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.65 avg-zval = 33.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927883 bytes = 3247600 } seq expand size = 6495181 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5154 value = 3.5326 dict = 1.00 all = 3.6534 } Zip/UnZip{ index = 0.0951 value = 0.2831 dict = 1.00 all = 0.2737 } ---------------------------- total value len = 0.803440 GB avg = 0.124 KB (by entry num) total key len = 0.181445 GB avg = 0.028 KB total ukey len = 0.129604 GB avg = 0.020 KB total ukey num = 0.006480191 Billion total entry num = 0.006480191 Billion write speed all = 0.019182074 MB/sec (with seq num) write speed all = 0.018172385 MB/sec (without seq num) 2024/11/22-23:08:07.207754 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232973: 927883 keys, 36738004 bytes, temperature: kWarm 2024/11/22-23:08:07.207842 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288087207798, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232973, "file_size": 36738004, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3631594159, "table_properties": {"data_size": 30639808, "index_size": 1764800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25980724, "raw_average_key_size": 28, "raw_value_size": 108237042, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927883, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288083, "oldest_key_time": 0, "file_creation_time": 1732288083, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232973, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:08:07.274550 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232974.sst) /dev/shm/ToplingTemp/Topling-326652-QEGhQ6 2024/11/22-23:08:08.291071 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232974.sst): first pass time = 1.02's, 132.043'MB/sec 2024/11/22-23:08:09.765685 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232974.sst): old prealloc_size = 147639500, real_size = 31482272 2024/11/22-23:08:09.766241 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232974.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924856 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0295 GB avg-val = 117.12 avg-zval = 31.89 2024/11/22-23:08:09.767021 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232974.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:08:10.629366 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232974.sst): second pass time = 1.47's, 73.518'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.191'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3293.974'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 279.999'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 39039.211'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.518'MB/sec zip pipeline throughput = 73.518'MB/sec entries = 924856 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.12 avg-zval = 31.89 usrkeys = 924856 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.12 avg-zval = 31.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924856 bytes = 3237008 } seq expand size = 6473992 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5124 value = 3.6730 dict = 1.00 all = 3.7728 } Zip/UnZip{ index = 0.0951 value = 0.2723 dict = 1.00 all = 0.2651 } ---------------------------- total value len = 0.918236 GB avg = 0.124 KB (by entry num) total key len = 0.207341 GB avg = 0.028 KB total ukey len = 0.148101 GB avg = 0.020 KB total ukey num = 0.007405047 Billion total entry num = 0.007405047 Billion write speed all = 0.021920773 MB/sec (with seq num) write speed all = 0.020767058 MB/sec (without seq num) 2024/11/22-23:08:10.662256 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232974: 924856 keys, 35575134 bytes, temperature: kWarm 2024/11/22-23:08:10.662318 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288090662285, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232974, "file_size": 35575134, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3450761200, "largest_seqno": 3631594299, "table_properties": {"data_size": 29491344, "index_size": 1759552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25895968, "raw_average_key_size": 28, "raw_value_size": 108321804, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924856, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288087, "oldest_key_time": 0, "file_creation_time": 1732288087, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232974, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:08:10.697407 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232975.sst) /dev/shm/ToplingTemp/Topling-326652-3HCSB3 2024/11/22-23:08:11.496300 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232975.sst): first pass time = 0.80's, 168.013'MB/sec 2024/11/22-23:08:12.844490 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232975.sst): old prealloc_size = 147639500, real_size = 32441056 2024/11/22-23:08:12.845052 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232975.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925124 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0304 GB avg-val = 117.08 avg-zval = 32.90 2024/11/22-23:08:12.845857 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232975.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:08:13.732557 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232975.sst): second pass time = 1.35's, 80.379'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 41.275'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3285.990'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 282.761'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36129.545'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.379'MB/sec zip pipeline throughput = 80.379'MB/sec entries = 925124 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.08 avg-zval = 32.90 usrkeys = 925124 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.08 avg-zval = 32.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925124 bytes = 3353584 } seq expand size = 6475868 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4444 value = 3.5585 dict = 1.00 all = 3.6617 } Zip/UnZip{ index = 0.0957 value = 0.2810 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 1.033027 GB avg = 0.124 KB (by entry num) total key len = 0.233245 GB avg = 0.028 KB total ukey len = 0.166603 GB avg = 0.020 KB total ukey num = 0.008330171 Billion total entry num = 0.008330171 Billion write speed all = 0.024659311 MB/sec (with seq num) write speed all = 0.023361540 MB/sec (without seq num) 2024/11/22-23:08:13.768894 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232975: 925124 keys, 36654196 bytes, temperature: kWarm 2024/11/22-23:08:13.768958 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288093768927, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232975, "file_size": 36654196, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3631592581, "table_properties": {"data_size": 30438096, "index_size": 1771520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903472, "raw_average_key_size": 28, "raw_value_size": 108314352, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925124, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288090, "oldest_key_time": 0, "file_creation_time": 1732288090, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232975, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:08:13.806698 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232976.sst) /dev/shm/ToplingTemp/Topling-326652-gSzup7 2024/11/22-23:08:14.564086 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232976.sst): first pass time = 0.76's, 177.223'MB/sec 2024/11/22-23:08:16.043276 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232976.sst): old prealloc_size = 147639500, real_size = 37202272 2024/11/22-23:08:16.043823 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232976.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925820 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0352 GB avg-val = 116.97 avg-zval = 38.03 2024/11/22-23:08:16.044182 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232976.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:08:16.734325 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232976.sst): second pass time = 1.48's, 73.265'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.69's, 56.743'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3360.041'MB/sec (index lex order gen) rebuild zvType time = 0.56's, 0.411'MB/sec write SST data time = 0.13's, 308.560'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33859.860'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.265'MB/sec zip pipeline throughput = 73.265'MB/sec entries = 925820 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 38.03 usrkeys = 925820 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.31 avg-zval = 38.03 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 309552 bytes = 1006064 } seq expand size = 2166864 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0352 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5134 value = 3.0757 dict = 1.00 all = 3.4229 } Zip/UnZip{ index = 0.0951 value = 0.3251 dict = 1.00 all = 0.2921 } ---------------------------- total value len = 1.143488 GB avg = 0.124 KB (by entry num) total key len = 0.259168 GB avg = 0.028 KB total ukey len = 0.185120 GB avg = 0.020 KB total ukey num = 0.009255991 Billion total entry num = 0.009255991 Billion write speed all = 0.027313663 MB/sec (with seq num) write speed all = 0.025871741 MB/sec (without seq num) 2024/11/22-23:08:16.769303 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232976: 925820 keys, 39211653 bytes, temperature: kWarm 2024/11/22-23:08:16.769366 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288096769335, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232976, "file_size": 39211653, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 35209424, "index_size": 1761216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25922960, "raw_average_key_size": 28, "raw_value_size": 108294840, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925820, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288093, "oldest_key_time": 0, "file_creation_time": 1732288093, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232976, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:08:16.808542 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232977.sst) /dev/shm/ToplingTemp/Topling-326652-qVoVR3 2024/11/22-23:08:17.540264 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232977.sst): first pass time = 0.73's, 183.437'MB/sec 2024/11/22-23:08:19.125342 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232977.sst): old prealloc_size = 147639500, real_size = 36452384 2024/11/22-23:08:19.125891 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232977.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927066 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.78 avg-zval = 37.18 2024/11/22-23:08:19.125933 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232977.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:08:19.765035 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232977.sst): second pass time = 1.58's, 68.330'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.327'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3304.358'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.451'MB/sec write SST data time = 0.13's, 296.947'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35619.465'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.330'MB/sec zip pipeline throughput = 68.330'MB/sec entries = 927066 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.78 avg-zval = 37.18 usrkeys = 927066 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.78 avg-zval = 37.18 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5818 value = 3.1409 dict = 1.00 all = 3.5973 } Zip/UnZip{ index = 0.0945 value = 0.3184 dict = 1.00 all = 0.2780 } ---------------------------- total value len = 1.251748 GB avg = 0.123 KB (by entry num) total key len = 0.285126 GB avg = 0.028 KB total ukey len = 0.203661 GB avg = 0.020 KB total ukey num = 0.010183057 Billion total entry num = 0.010183057 Billion write speed all = 0.029925495 MB/sec (with seq num) write speed all = 0.028339246 MB/sec (without seq num) 2024/11/22-23:08:19.799955 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232977: 927066 keys, 37310456 bytes, temperature: kWarm 2024/11/22-23:08:19.800019 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288099799986, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232977, "file_size": 37310456, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34468240, "index_size": 1752192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957848, "raw_average_key_size": 28, "raw_value_size": 108259920, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927066, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288096, "oldest_key_time": 0, "file_creation_time": 1732288096, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232977, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:08:19.837796 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232978.sst) /dev/shm/ToplingTemp/Topling-326652-JNFP27 2024/11/22-23:08:20.731740 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232978.sst): first pass time = 0.89's, 150.149'MB/sec 2024/11/22-23:08:22.176756 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232978.sst): old prealloc_size = 147639500, real_size = 36573920 2024/11/22-23:08:22.177297 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232978.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924629 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0346 GB avg-val = 117.16 avg-zval = 37.40 2024/11/22-23:08:22.177610 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232978.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:08:22.896940 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232978.sst): second pass time = 1.44's, 75.003'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 53.489'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3486.870'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.390'MB/sec write SST data time = 0.13's, 303.208'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33662.257'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.003'MB/sec zip pipeline throughput = 75.003'MB/sec entries = 924629 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.16 avg-zval = 37.40 usrkeys = 924629 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.36 avg-zval = 37.40 TagRS{ kinds = RS_Key0_TagN bytes = 144560 } TagArray{ size = 290192 bytes = 943136 } seq expand size = 2031344 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5121 value = 3.1324 dict = 1.00 all = 3.4842 } Zip/UnZip{ index = 0.0951 value = 0.3192 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 1.362108 GB avg = 0.123 KB (by entry num) total key len = 0.311015 GB avg = 0.028 KB total ukey len = 0.222154 GB avg = 0.020 KB total ukey num = 0.011107686 Billion total entry num = 0.011107686 Billion write speed all = 0.032576506 MB/sec (with seq num) write speed all = 0.030846330 MB/sec (without seq num) 2024/11/22-23:08:22.932685 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232978: 924629 keys, 38521415 bytes, temperature: kWarm 2024/11/22-23:08:22.932747 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288102932716, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232978, "file_size": 38521415, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3333204587, "table_properties": {"data_size": 34583440, "index_size": 1759168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25889612, "raw_average_key_size": 28, "raw_value_size": 108328190, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924629, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288099, "oldest_key_time": 0, "file_creation_time": 1732288099, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232978, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:08:22.968253 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232979.sst) /dev/shm/ToplingTemp/Topling-326652-Fmbxh6 2024/11/22-23:08:23.877168 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232979.sst): first pass time = 0.91's, 147.675'MB/sec 2024/11/22-23:08:25.208288 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232979.sst): old prealloc_size = 147639500, real_size = 32689680 2024/11/22-23:08:25.208848 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232979.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925383 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 117.04 avg-zval = 33.17 2024/11/22-23:08:25.209641 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232979.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:08:26.108330 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232979.sst): second pass time = 1.33's, 81.404'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.998'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3265.192'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.13's, 283.209'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 21364.744'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.404'MB/sec zip pipeline throughput = 81.404'MB/sec entries = 925383 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 33.17 usrkeys = 925383 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 33.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925383 bytes = 3354528 } seq expand size = 6477681 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5130 value = 3.5282 dict = 1.00 all = 3.6374 } Zip/UnZip{ index = 0.0951 value = 0.2834 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 1.476892 GB avg = 0.123 KB (by entry num) total key len = 0.336926 GB avg = 0.028 KB total ukey len = 0.240661 GB avg = 0.020 KB total ukey num = 0.012033069 Billion total entry num = 0.012033069 Billion write speed all = 0.035313707 MB/sec (with seq num) write speed all = 0.033439507 MB/sec (without seq num) 2024/11/22-23:08:26.140877 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232979: 925383 keys, 36899290 bytes, temperature: kWarm 2024/11/22-23:08:26.140939 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288106140907, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232979, "file_size": 36899290, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3333204588, "largest_seqno": 3631594591, "table_properties": {"data_size": 30697728, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25910724, "raw_average_key_size": 28, "raw_value_size": 108307066, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925383, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288102, "oldest_key_time": 0, "file_creation_time": 1732288102, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232979, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:08:26.177039 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232980.sst) /dev/shm/ToplingTemp/Topling-326652-UQRGj6 2024/11/22-23:08:27.052527 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232980.sst): first pass time = 0.88's, 153.314'MB/sec 2024/11/22-23:08:28.404797 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232980.sst): old prealloc_size = 147639500, real_size = 33099600 2024/11/22-23:08:28.405320 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232980.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929311 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.43 avg-zval = 33.46 2024/11/22-23:08:28.406058 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232980.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:08:29.352559 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232980.sst): second pass time = 1.35's, 80.052'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.011'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3517.970'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.284'MB/sec write SST data time = 0.13's, 287.718'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35254.572'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.052'MB/sec zip pipeline throughput = 80.052'MB/sec entries = 929311 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.43 avg-zval = 33.46 usrkeys = 929311 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.43 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929311 bytes = 3020272 } seq expand size = 6505177 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5137 value = 3.4791 dict = 1.00 all = 3.6302 } Zip/UnZip{ index = 0.0951 value = 0.2874 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 1.591595 GB avg = 0.123 KB (by entry num) total key len = 0.362947 GB avg = 0.028 KB total ukey len = 0.259248 GB avg = 0.020 KB total ukey num = 0.012962380 Billion total entry num = 0.012962380 Billion write speed all = 0.038051075 MB/sec (with seq num) write speed all = 0.036032258 MB/sec (without seq num) 2024/11/22-23:08:29.386800 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232980: 929311 keys, 36972964 bytes, temperature: kWarm 2024/11/22-23:08:29.386862 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288109386830, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232980, "file_size": 36972964, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3572778513, "largest_seqno": 3631594545, "table_properties": {"data_size": 31099264, "index_size": 1767808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26020708, "raw_average_key_size": 28, "raw_value_size": 108197042, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929311, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288106, "oldest_key_time": 0, "file_creation_time": 1732288106, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232980, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:08:29.422693 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232981.sst) /dev/shm/ToplingTemp/Topling-326652-wV0fM5 2024/11/22-23:08:30.374719 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232981.sst): first pass time = 0.95's, 140.987'MB/sec 2024/11/22-23:08:31.706126 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232981.sst): old prealloc_size = 147639500, real_size = 34668704 2024/11/22-23:08:31.706710 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232981.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931797 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0327 GB avg-val = 116.04 avg-zval = 35.04 2024/11/22-23:08:31.707511 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232981.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-23:08:32.597606 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232981.sst): second pass time = 1.33's, 81.297'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.633'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3172.301'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 303.518'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35179.131'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.297'MB/sec zip pipeline throughput = 81.297'MB/sec entries = 931797 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.04 avg-zval = 35.04 usrkeys = 931797 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.04 avg-zval = 35.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931797 bytes = 3377776 } seq expand size = 6522579 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4435 value = 3.3116 dict = 1.00 all = 3.4505 } Zip/UnZip{ index = 0.0958 value = 0.3020 dict = 1.00 all = 0.2898 } ---------------------------- total value len = 1.706245 GB avg = 0.123 KB (by entry num) total key len = 0.389037 GB avg = 0.028 KB total ukey len = 0.277884 GB avg = 0.020 KB total ukey num = 0.013894177 Billion total entry num = 0.013894177 Billion write speed all = 0.040788437 MB/sec (with seq num) write speed all = 0.038624635 MB/sec (without seq num) 2024/11/22-23:08:32.631985 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232981: 931797 keys, 38897514 bytes, temperature: kWarm 2024/11/22-23:08:32.632046 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288112632014, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232981, "file_size": 38897514, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3631594474, "table_properties": {"data_size": 32651152, "index_size": 1784448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26090316, "raw_average_key_size": 28, "raw_value_size": 108127494, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931797, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288109, "oldest_key_time": 0, "file_creation_time": 1732288109, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232981, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:08:32.667861 140024709862976 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232982.sst) /dev/shm/ToplingTemp/Topling-326652-fPe7v3 2024/11/22-23:08:32.747154 140024709862976 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232982.sst): first pass time = 0.08's, 237.672'MB/sec 2024/11/22-23:08:32.922593 140024709862976 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232982.sst): old prealloc_size = 147639500, real_size = 5657680 2024/11/22-23:08:32.922729 140024709862976 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232982.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 130051 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0152 GB zip-val = 0.0054 GB avg-val = 116.85 avg-zval = 41.34 2024/11/22-23:08:32.922880 140024709862976 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232982.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.019 GB 2024/11/22-23:08:33.044496 140024709862976 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00/232982.sst): second pass time = 0.17's, 86.895'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 51.206'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2133.224'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.336'MB/sec write SST data time = 0.02's, 250.790'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.159'MB, throughput = 5914.876'MB/sec zip my value time = 0.17's, unzip length = 0.015'GB zip my value throughput = 86.895'MB/sec zip pipeline throughput = 86.895'MB/sec entries = 130051 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.85 avg-zval = 41.34 usrkeys = 130051 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.85 avg-zval = 41.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 130051 bytes = 455200 } seq expand size = 910357 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0026 GB value = 0.0152 GB dict = 0.16 MB all = 0.0188 GB } __ZipSize{ index = 0.0002 GB value = 0.0054 GB dict = 0.16 MB all = 0.0062 GB } UnZip/Zip{ index = 10.4610 value = 2.8264 dict = 1.00 all = 3.0182 } Zip/UnZip{ index = 0.0956 value = 0.3538 dict = 1.00 all = 0.3313 } ---------------------------- total value len = 1.722351 GB avg = 0.123 KB (by entry num) total key len = 0.392678 GB avg = 0.028 KB total ukey len = 0.280485 GB avg = 0.020 KB total ukey num = 0.014024228 Billion total entry num = 0.014024228 Billion write speed all = 0.041172502 MB/sec (with seq num) write speed all = 0.038988466 MB/sec (without seq num) 2024/11/22-23:08:33.056095 140024709862976 [compaction_job.cc:1948] [default] [JOB 13556] Generated table #232982: 130051 keys, 6241313 bytes, temperature: kWarm 2024/11/22-23:08:33.056150 140024709862976 EVENT_LOG_v1 {"time_micros": 1732288113056122, "cf_name": "default", "job": 13556, "event": "table_file_creation", "file_number": 232982, "file_size": 6241313, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3630099684, "table_properties": {"data_size": 5376320, "index_size": 248640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3641428, "raw_average_key_size": 28, "raw_value_size": 15195874, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 130051, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732288112, "oldest_key_time": 0, "file_creation_time": 1732288112, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232982, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-23:08:33.068133 140024709862976 [compaction_job.cc:703] job-13556: subcompact[0], size: 0.564889 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13556/att-00 [232967,232968,232969,232970,232971,232972,232973,232974,232975,232976,232977,232978,232979,232980,232981,232982] 2024/11/22-23:08:33.168434 140024709862976 (Original Log Time 2024/11/22-23:07:35.243426) EVENT_LOG_v1 {"time_micros": 1732288055243370, "job": 13556, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [232963], "files_L3": [232911, 232912, 232913, 232914, 232915, 232916, 232917, 232918, 232919, 232920, 232921, 232922, 232923, 232924, 232927], "score": 0.998627, "input_data_size": 588183146, "oldest_snapshot_seqno": 3631612166} 2024/11/22-23:08:33.168437 140024709862976 (Original Log Time 2024/11/22-23:08:33.167600) [compaction_job.cc:2022] [default] [JOB 13556] Compacted 1@2 + 15@3 files to L3 => 564889349 bytes 2024/11/22-23:08:33.168439 140024709862976 (Original Log Time 2024/11/22-23:08:33.168361) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.2 rd, 9.8 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.1, 533.8 +0.0 blob) out(538.7 +0.0 blob), read-write-amplify(40.6) write-amplify(19.9) OK, records in: 14127459, records dropped: 103231 output_compression: Snappy 2024/11/22-23:08:33.168443 140024709862976 (Original Log Time 2024/11/22-23:08:33.168414) EVENT_LOG_v1 {"time_micros": 1732288113168385, "cf": "default", "job": 13556, "event": "compaction_finished", "compaction_time_micros": 57824714, "compaction_time_cpu_micros": 23601027, "output_level": 3, "num_output_files": 16, "total_output_size": 564889349, "num_input_records": 14127459, "num_output_records": 14024228, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 386816654, "file_range_sync_nanos": 0, "file_fsync_nanos": 14478, "file_prepare_write_nanos": 61710, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/22-23:08:33.298288 140024709862976 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732288055,"executesMs":58044,"compactionJobId":13556,"attempt":0,"compactionInputRawBytes":2046762922,"compactionInputZipBytes":588183146,"compactionOutputRawBytes":0,"compactionOutputZipBytes":564889349,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-b47939842f6075b9b30d5bbdbf4854f7-c542bfff251c72cc-00"}}, response = 2024/11/22-23:08:33.298367 140024709862976 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13556/att-00: olev 3, work 58.044 s, result 6.369 ms, install 0.864 ms, input{raw 1.906 GiB zip 560.935 MiB}