2024/11/23-02:43:15.131749 140025104320064 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00: fork to child time = 0.008928 sec 2024/11/23-02:43:15.132687 140025104320064 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00 2024/11/23-02:43:15.138981 140025104320064 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00 2024/11/23-02:43:15.140103 140025104320064 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-02:43:15.140378 140025104320064 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-02:43:15.195973 140025104320064 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-17633/att-00/MANIFEST-206347 2024/11/23-02:43:15.225876 140025104320064 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-17633/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 241257, last_sequence is 3740277375, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 241112 2024/11/23-02:43:15.225882 140025104320064 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-02:43:15.225993 140025104320064 [version_set.cc:5721] Creating manifest 241257 2024/11/23-02:43:15.260032 140025104320064 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-02:43:15.282637 140025104320064 [compaction_job.cc:2437] [default] [JOB 17633] Compacting 1@2 + 16@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-02:43:15.282651 140025104320064 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 13174 Base level 2, inputs: [241192(27MB)], [241249(35MB) 241250(5291KB) 241157(35MB) 241158(34MB) 241159(11MB) 241120(35MB) 241121(35MB) 241122(35MB) 241123(35MB) 241124(35MB) 241125(34MB) 241126(34MB) 241127(35MB) 241134(35MB) 241135(34MB) 241227(36MB)] 2024/11/23-02:43:15.288502 140025104320064 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-02:43:15.288534 140025104320064 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-02:43:15.288554 140025104320064 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-02:43:15.288559 140025104320064 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-02:43:15.288569 140025104320064 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-02:43:15.937597 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241258.sst) /dev/shm/ToplingTemp/Topling-384761-OKLcX0 2024/11/23-02:43:16.478490 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241258.sst): first pass time = 0.54's, 248.161'MB/sec 2024/11/23-02:43:17.804530 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241258.sst): old prealloc_size = 147639500, real_size = 33005440 2024/11/23-02:43:17.805098 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241258.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926955 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.79 avg-zval = 33.45 2024/11/23-02:43:17.805899 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241258.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:18.724598 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241258.sst): second pass time = 1.32's, 81.723'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.454'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3224.983'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 291.040'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33586.880'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.723'MB/sec zip pipeline throughput = 81.723'MB/sec entries = 926955 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 33.45 usrkeys = 926955 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.79 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926955 bytes = 3360224 } seq expand size = 6488685 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5106 value = 3.4913 dict = 1.00 all = 3.6061 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2773 } ---------------------------- total value len = 0.114752 GB avg = 0.124 KB (by entry num) total key len = 0.025955 GB avg = 0.028 KB total ukey len = 0.018539 GB avg = 0.020 KB total ukey num = 0.000926955 Billion total entry num = 0.000926955 Billion write speed all = 0.002189792 MB/sec (with seq num) write speed all = 0.002074384 MB/sec (without seq num) 2024/11/23-02:43:18.759206 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241258: 926955 keys, 37220050 bytes, temperature: kWarm 2024/11/23-02:43:18.759290 140025104320064 EVENT_LOG_v1 {"time_micros": 1732300998759238, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241258, "file_size": 37220050, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3739845117, "table_properties": {"data_size": 31009648, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954740, "raw_average_key_size": 28, "raw_value_size": 108263066, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926955, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732300995, "oldest_key_time": 0, "file_creation_time": 1732300995, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241258, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:18.794240 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241259.sst) /dev/shm/ToplingTemp/Topling-384761-ew9851 2024/11/23-02:43:19.739996 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241259.sst): first pass time = 0.95's, 141.920'MB/sec 2024/11/23-02:43:21.076663 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241259.sst): old prealloc_size = 147639500, real_size = 32320864 2024/11/23-02:43:21.077268 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241259.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927177 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 116.76 avg-zval = 32.68 2024/11/23-02:43:21.078039 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241259.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:21.979133 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241259.sst): second pass time = 1.34's, 81.030'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.357'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3068.107'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.299'MB/sec write SST data time = 0.13's, 287.119'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31519.939'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.030'MB/sec zip pipeline throughput = 81.030'MB/sec entries = 927177 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.76 avg-zval = 32.68 usrkeys = 927177 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.76 avg-zval = 32.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927177 bytes = 3245136 } seq expand size = 6490239 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.3757 value = 3.5726 dict = 1.00 all = 3.6852 } Zip/UnZip{ index = 0.0964 value = 0.2799 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 0.229499 GB avg = 0.124 KB (by entry num) total key len = 0.051916 GB avg = 0.028 KB total ukey len = 0.037083 GB avg = 0.020 KB total ukey num = 0.001854132 Billion total entry num = 0.001854132 Billion write speed all = 0.004379386 MB/sec (with seq num) write speed all = 0.004148553 MB/sec (without seq num) 2024/11/23-02:43:22.014726 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241259: 927177 keys, 36420584 bytes, temperature: kWarm 2024/11/23-02:43:22.014806 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301002014757, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241259, "file_size": 36420584, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3739844670, "table_properties": {"data_size": 30301712, "index_size": 1787200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960956, "raw_average_key_size": 28, "raw_value_size": 108256782, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927177, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732300998, "oldest_key_time": 0, "file_creation_time": 1732300998, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241259, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:22.060870 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241260.sst) /dev/shm/ToplingTemp/Topling-384761-Cghl00 2024/11/23-02:43:22.909516 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241260.sst): first pass time = 0.85's, 158.162'MB/sec 2024/11/23-02:43:24.236465 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241260.sst): old prealloc_size = 147639500, real_size = 30898112 2024/11/23-02:43:24.237054 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241260.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927167 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 116.76 avg-zval = 31.17 2024/11/23-02:43:24.237770 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241260.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:25.095347 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241260.sst): second pass time = 1.33's, 81.626'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 40.614'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3099.526'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 274.261'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 30135.207'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.626'MB/sec zip pipeline throughput = 81.626'MB/sec entries = 927167 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 31.17 usrkeys = 927167 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.76 avg-zval = 31.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927167 bytes = 3129200 } seq expand size = 6490169 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.5134 value = 3.7456 dict = 1.00 all = 3.8478 } Zip/UnZip{ index = 0.0951 value = 0.2670 dict = 1.00 all = 0.2599 } ---------------------------- total value len = 0.344246 GB avg = 0.124 KB (by entry num) total key len = 0.077876 GB avg = 0.028 KB total ukey len = 0.055626 GB avg = 0.020 KB total ukey num = 0.002781299 Billion total entry num = 0.002781299 Billion write speed all = 0.006568773 MB/sec (with seq num) write speed all = 0.006222528 MB/sec (without seq num) 2024/11/23-02:43:25.124932 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241260: 927167 keys, 34881888 bytes, temperature: kWarm 2024/11/23-02:43:25.124994 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301005124961, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241260, "file_size": 34881888, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3614942400, "largest_seqno": 3739844690, "table_properties": {"data_size": 28902384, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960676, "raw_average_key_size": 28, "raw_value_size": 108257162, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927167, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301002, "oldest_key_time": 0, "file_creation_time": 1732301002, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241260, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:25.160157 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241261.sst) /dev/shm/ToplingTemp/Topling-384761-g43Uy0 2024/11/23-02:43:26.054755 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241261.sst): first pass time = 0.89's, 150.036'MB/sec 2024/11/23-02:43:27.381006 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241261.sst): old prealloc_size = 147639500, real_size = 31646656 2024/11/23-02:43:27.381571 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241261.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926929 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 116.80 avg-zval = 31.99 2024/11/23-02:43:27.382339 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241261.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:28.247540 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241261.sst): second pass time = 1.33's, 81.675'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.385'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3246.384'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 280.630'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31647.955'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.675'MB/sec zip pipeline throughput = 81.675'MB/sec entries = 926929 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 31.99 usrkeys = 926929 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926929 bytes = 3360128 } seq expand size = 6488503 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5138 value = 3.6512 dict = 1.00 all = 3.7427 } Zip/UnZip{ index = 0.0951 value = 0.2739 dict = 1.00 all = 0.2672 } ---------------------------- total value len = 0.458998 GB avg = 0.124 KB (by entry num) total key len = 0.103830 GB avg = 0.028 KB total ukey len = 0.074165 GB avg = 0.020 KB total ukey num = 0.003708228 Billion total entry num = 0.003708228 Billion write speed all = 0.008757915 MB/sec (with seq num) write speed all = 0.008296299 MB/sec (without seq num) 2024/11/23-02:43:28.282162 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241261: 926929 keys, 35861152 bytes, temperature: kWarm 2024/11/23-02:43:28.282224 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301008282191, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241261, "file_size": 35861152, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3739844703, "table_properties": {"data_size": 29651504, "index_size": 1763264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954012, "raw_average_key_size": 28, "raw_value_size": 108263762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926929, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301005, "oldest_key_time": 0, "file_creation_time": 1732301005, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241261, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:28.317934 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241262.sst) /dev/shm/ToplingTemp/Topling-384761-18XuEZ 2024/11/23-02:43:29.206458 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241262.sst): first pass time = 0.89's, 151.063'MB/sec 2024/11/23-02:43:30.554327 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241262.sst): old prealloc_size = 147639500, real_size = 34335872 2024/11/23-02:43:30.554929 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241262.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931353 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0323 GB avg-val = 116.11 avg-zval = 34.71 2024/11/23-02:43:30.555665 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241262.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:31.468036 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241262.sst): second pass time = 1.35's, 80.307'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.078'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3049.891'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.296'MB/sec write SST data time = 0.13's, 301.701'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32700.209'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.307'MB/sec zip pipeline throughput = 80.307'MB/sec entries = 931353 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.11 avg-zval = 34.71 usrkeys = 931353 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.11 avg-zval = 34.71 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931353 bytes = 3259744 } seq expand size = 6519471 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5159 value = 3.3447 dict = 1.00 all = 3.4910 } Zip/UnZip{ index = 0.0951 value = 0.2990 dict = 1.00 all = 0.2865 } ---------------------------- total value len = 0.573658 GB avg = 0.124 KB (by entry num) total key len = 0.129908 GB avg = 0.028 KB total ukey len = 0.092792 GB avg = 0.020 KB total ukey num = 0.004639581 Billion total entry num = 0.004639581 Billion write speed all = 0.010947313 MB/sec (with seq num) write speed all = 0.010369787 MB/sec (without seq num) 2024/11/23-02:43:31.500513 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241262: 931353 keys, 38446816 bytes, temperature: kWarm 2024/11/23-02:43:31.500575 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301011500542, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241262, "file_size": 38446816, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3739844969, "table_properties": {"data_size": 32331504, "index_size": 1771328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26077884, "raw_average_key_size": 28, "raw_value_size": 108139874, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931353, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301008, "oldest_key_time": 0, "file_creation_time": 1732301008, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241262, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:31.536734 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241263.sst) /dev/shm/ToplingTemp/Topling-384761-iK3Ep1 2024/11/23-02:43:32.408815 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241263.sst): first pass time = 0.87's, 153.911'MB/sec 2024/11/23-02:43:33.745461 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241263.sst): old prealloc_size = 147639500, real_size = 32348720 2024/11/23-02:43:33.746058 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241263.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926992 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 116.79 avg-zval = 32.72 2024/11/23-02:43:33.746798 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241263.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:34.638087 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241263.sst): second pass time = 1.34's, 81.063'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.833'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3103.446'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.303'MB/sec write SST data time = 0.13's, 286.810'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31824.819'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.063'MB/sec zip pipeline throughput = 81.063'MB/sec entries = 926992 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.79 avg-zval = 32.72 usrkeys = 926992 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.79 avg-zval = 32.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926992 bytes = 3244480 } seq expand size = 6488944 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.3796 value = 3.5694 dict = 1.00 all = 3.6825 } Zip/UnZip{ index = 0.0963 value = 0.2802 dict = 1.00 all = 0.2716 } ---------------------------- total value len = 0.688409 GB avg = 0.124 KB (by entry num) total key len = 0.155864 GB avg = 0.028 KB total ukey len = 0.111331 GB avg = 0.020 KB total ukey num = 0.005566573 Billion total entry num = 0.005566573 Billion write speed all = 0.013136028 MB/sec (with seq num) write speed all = 0.012443146 MB/sec (without seq num) 2024/11/23-02:43:34.673966 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241263: 926992 keys, 36447574 bytes, temperature: kWarm 2024/11/23-02:43:34.674030 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301014673996, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241263, "file_size": 36447574, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3587898319, "largest_seqno": 3739844749, "table_properties": {"data_size": 30330592, "index_size": 1786176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955776, "raw_average_key_size": 28, "raw_value_size": 108262088, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926992, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301011, "oldest_key_time": 0, "file_creation_time": 1732301011, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241263, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:34.713788 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241264.sst) /dev/shm/ToplingTemp/Topling-384761-INk882 2024/11/23-02:43:35.623630 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241264.sst): first pass time = 0.91's, 147.523'MB/sec 2024/11/23-02:43:36.890053 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241264.sst): old prealloc_size = 147639500, real_size = 31237264 2024/11/23-02:43:36.890631 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241264.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927659 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0292 GB avg-val = 116.68 avg-zval = 31.52 2024/11/23-02:43:36.891341 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241264.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:37.768725 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241264.sst): second pass time = 1.27's, 85.517'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.060'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3158.932'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.13's, 267.853'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34645.008'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.517'MB/sec zip pipeline throughput = 85.517'MB/sec entries = 927659 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 31.52 usrkeys = 927659 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.30 avg-zval = 31.52 TagRS{ kinds = RS_Key0_TagN bytes = 145040 } TagArray{ size = 876483 bytes = 2958144 } seq expand size = 6135381 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.5156 value = 3.7018 dict = 1.00 all = 3.8132 } Zip/UnZip{ index = 0.0951 value = 0.2701 dict = 1.00 all = 0.2622 } ---------------------------- total value len = 0.802787 GB avg = 0.124 KB (by entry num) total key len = 0.181838 GB avg = 0.028 KB total ukey len = 0.129885 GB avg = 0.020 KB total ukey num = 0.006494232 Billion total entry num = 0.006494232 Billion write speed all = 0.015319035 MB/sec (with seq num) write speed all = 0.014510726 MB/sec (without seq num) 2024/11/23-02:43:37.799522 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241264: 927659 keys, 35198618 bytes, temperature: kWarm 2024/11/23-02:43:37.799594 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301017799561, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241264, "file_size": 35198618, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3739844839, "table_properties": {"data_size": 29240832, "index_size": 1764352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25974452, "raw_average_key_size": 28, "raw_value_size": 108243366, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927659, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301014, "oldest_key_time": 0, "file_creation_time": 1732301014, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241264, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:37.834971 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241265.sst) /dev/shm/ToplingTemp/Topling-384761-GvcSf3 2024/11/23-02:43:38.727454 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241265.sst): first pass time = 0.89's, 150.394'MB/sec 2024/11/23-02:43:40.110372 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241265.sst): old prealloc_size = 147639500, real_size = 34981280 2024/11/23-02:43:40.111064 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241265.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081570 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.10 avg-zval = 30.18 2024/11/23-02:43:40.111108 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241265.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:40.735997 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241265.sst): second pass time = 1.38's, 75.197'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.63's, 57.150'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3087.913'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.556'MB/sec write SST data time = 0.14's, 257.533'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.031'MB, throughput = 22111.588'MB/sec zip my value time = 1.38's, unzip length = 0.104'GB zip my value throughput = 75.197'MB/sec zip pipeline throughput = 75.197'MB/sec entries = 1081570 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.10 avg-zval = 30.18 usrkeys = 1081570 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.10 avg-zval = 30.18 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.03 MB all = 0.0358 GB } UnZip/Zip{ index = 10.4476 value = 3.1842 dict = 1.00 all = 3.7540 } Zip/UnZip{ index = 0.0957 value = 0.3140 dict = 1.00 all = 0.2664 } ---------------------------- total value len = 0.906721 GB avg = 0.120 KB (by entry num) total key len = 0.212122 GB avg = 0.028 KB total ukey len = 0.151516 GB avg = 0.020 KB total ukey num = 0.007575802 Billion total entry num = 0.007575802 Billion write speed all = 0.017406423 MB/sec (with seq num) write speed all = 0.016463538 MB/sec (without seq num) 2024/11/23-02:43:40.770178 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241265: 1081570 keys, 35753381 bytes, temperature: kWarm 2024/11/23-02:43:40.770236 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301020770205, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241265, "file_size": 35753381, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32640208, "index_size": 2070464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30283960, "raw_average_key_size": 28, "raw_value_size": 103933849, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081570, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301017, "oldest_key_time": 0, "file_creation_time": 1732301017, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241265, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:40.805171 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241266.sst) /dev/shm/ToplingTemp/Topling-384761-xd9RA1 2024/11/23-02:43:41.687028 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241266.sst): first pass time = 0.88's, 152.204'MB/sec 2024/11/23-02:43:43.080863 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241266.sst): old prealloc_size = 147639500, real_size = 34033840 2024/11/23-02:43:43.081426 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241266.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921366 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0320 GB avg-val = 117.67 avg-zval = 34.77 2024/11/23-02:43:43.082165 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241266.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:43.953211 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241266.sst): second pass time = 1.39's, 77.826'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.936'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3261.377'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.311'MB/sec write SST data time = 0.13's, 286.833'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 30919.724'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 77.826'MB/sec zip pipeline throughput = 77.826'MB/sec entries = 921366 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.67 avg-zval = 34.77 usrkeys = 921366 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.73 avg-zval = 34.77 TagRS{ kinds = RS_Key0_TagN bytes = 144080 } TagArray{ size = 665709 bytes = 2413216 } seq expand size = 4659963 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4435 value = 3.3840 dict = 1.00 all = 3.5835 } Zip/UnZip{ index = 0.0958 value = 0.2955 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 1.019801 GB avg = 0.120 KB (by entry num) total key len = 0.237921 GB avg = 0.028 KB total ukey len = 0.169943 GB avg = 0.020 KB total ukey num = 0.008497168 Billion total entry num = 0.008497168 Billion write speed all = 0.019566035 MB/sec (with seq num) write speed all = 0.018508530 MB/sec (without seq num) 2024/11/23-02:43:43.987017 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241266: 921366 keys, 37454180 bytes, temperature: kWarm 2024/11/23-02:43:43.987083 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301023987047, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241266, "file_size": 37454180, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3738708733, "table_properties": {"data_size": 32038816, "index_size": 1764480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25798248, "raw_average_key_size": 28, "raw_value_size": 108419520, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921366, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301020, "oldest_key_time": 0, "file_creation_time": 1732301020, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241266, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:44.022431 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241267.sst) /dev/shm/ToplingTemp/Topling-384761-nfmvp1 2024/11/23-02:43:44.928559 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241267.sst): first pass time = 0.91's, 148.128'MB/sec 2024/11/23-02:43:46.225532 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241267.sst): old prealloc_size = 147639500, real_size = 32996656 2024/11/23-02:43:46.226107 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241267.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926987 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.79 avg-zval = 33.44 2024/11/23-02:43:46.226876 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241267.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:47.181336 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241267.sst): second pass time = 1.30's, 83.522'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 38.811'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.085'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.280'MB/sec write SST data time = 0.13's, 293.777'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32726.099'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.522'MB/sec zip pipeline throughput = 83.522'MB/sec entries = 926987 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 33.44 usrkeys = 926987 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.79 avg-zval = 33.44 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926987 bytes = 3244464 } seq expand size = 6488909 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5122 value = 3.4922 dict = 1.00 all = 3.6182 } Zip/UnZip{ index = 0.0951 value = 0.2864 dict = 1.00 all = 0.2764 } ---------------------------- total value len = 1.134552 GB avg = 0.120 KB (by entry num) total key len = 0.263876 GB avg = 0.028 KB total ukey len = 0.188483 GB avg = 0.020 KB total ukey num = 0.009424155 Billion total entry num = 0.009424155 Billion write speed all = 0.021753880 MB/sec (with seq num) write speed all = 0.020581066 MB/sec (without seq num) 2024/11/23-02:43:47.214666 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241267: 926987 keys, 37095178 bytes, temperature: kWarm 2024/11/23-02:43:47.214728 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301027214696, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241267, "file_size": 37095178, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3739844436, "table_properties": {"data_size": 31001056, "index_size": 1763648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955636, "raw_average_key_size": 28, "raw_value_size": 108262186, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926987, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301024, "oldest_key_time": 0, "file_creation_time": 1732301024, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241267, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:47.251911 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241268.sst) /dev/shm/ToplingTemp/Topling-384761-IGy6p0 2024/11/23-02:43:48.166515 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241268.sst): first pass time = 0.91's, 146.755'MB/sec 2024/11/23-02:43:49.509876 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241268.sst): old prealloc_size = 147639500, real_size = 33299440 2024/11/23-02:43:49.510456 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241268.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934741 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0313 GB avg-val = 115.59 avg-zval = 33.47 2024/11/23-02:43:49.511258 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241268.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:50.437810 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241268.sst): second pass time = 1.34's, 80.500'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 40.454'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3171.723'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.292'MB/sec write SST data time = 0.13's, 293.859'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30703.031'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.500'MB/sec zip pipeline throughput = 80.500'MB/sec entries = 934741 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.59 avg-zval = 33.47 usrkeys = 934741 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.59 avg-zval = 33.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934741 bytes = 3388448 } seq expand size = 6543187 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5116 value = 3.4533 dict = 1.00 all = 3.5755 } Zip/UnZip{ index = 0.0951 value = 0.2896 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 1.249140 GB avg = 0.121 KB (by entry num) total key len = 0.290049 GB avg = 0.028 KB total ukey len = 0.207178 GB avg = 0.020 KB total ukey num = 0.010358896 Billion total entry num = 0.010358896 Billion write speed all = 0.023942338 MB/sec (with seq num) write speed all = 0.022653264 MB/sec (without seq num) 2024/11/23-02:43:50.471426 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241268: 934741 keys, 37538034 bytes, temperature: kWarm 2024/11/23-02:43:50.471488 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301030471454, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241268, "file_size": 37538034, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3739844791, "table_properties": {"data_size": 31287072, "index_size": 1778496, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26172748, "raw_average_key_size": 28, "raw_value_size": 108045082, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934741, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301027, "oldest_key_time": 0, "file_creation_time": 1732301027, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241268, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:50.505542 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241269.sst) /dev/shm/ToplingTemp/Topling-384761-u04ZGZ 2024/11/23-02:43:51.421181 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241269.sst): first pass time = 0.92's, 146.589'MB/sec 2024/11/23-02:43:52.753327 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241269.sst): old prealloc_size = 147639500, real_size = 32593232 2024/11/23-02:43:52.753879 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241269.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927385 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 116.73 avg-zval = 32.98 2024/11/23-02:43:52.754632 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241269.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:53.662162 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241269.sst): second pass time = 1.33's, 81.332'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 40.247'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3421.247'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.13's, 286.451'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33260.867'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.332'MB/sec zip pipeline throughput = 81.332'MB/sec entries = 927385 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.73 avg-zval = 32.98 usrkeys = 927385 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.73 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927385 bytes = 3129936 } seq expand size = 6491695 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4435 value = 3.5393 dict = 1.00 all = 3.6694 } Zip/UnZip{ index = 0.0958 value = 0.2825 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 1.363883 GB avg = 0.121 KB (by entry num) total key len = 0.316016 GB avg = 0.028 KB total ukey len = 0.225726 GB avg = 0.020 KB total ukey num = 0.011286281 Billion total entry num = 0.011286281 Billion write speed all = 0.026129787 MB/sec (with seq num) write speed all = 0.024725378 MB/sec (without seq num) 2024/11/23-02:43:53.700324 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241269: 927385 keys, 36577486 bytes, temperature: kWarm 2024/11/23-02:43:53.700401 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301033700363, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241269, "file_size": 36577486, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3619876092, "largest_seqno": 3739843542, "table_properties": {"data_size": 30585216, "index_size": 1776000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25966780, "raw_average_key_size": 28, "raw_value_size": 108251046, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927385, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301030, "oldest_key_time": 0, "file_creation_time": 1732301030, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241269, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:53.736149 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241270.sst) /dev/shm/ToplingTemp/Topling-384761-4Lvnv2 2024/11/23-02:43:54.619385 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241270.sst): first pass time = 0.88's, 151.968'MB/sec 2024/11/23-02:43:55.970827 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241270.sst): old prealloc_size = 147639500, real_size = 33370464 2024/11/23-02:43:55.971368 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241270.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927093 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.77 avg-zval = 33.84 2024/11/23-02:43:55.972156 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241270.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:43:56.931382 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241270.sst): second pass time = 1.35's, 80.144'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 39.130'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3383.247'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.13's, 291.651'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33079.797'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.144'MB/sec zip pipeline throughput = 80.144'MB/sec entries = 927093 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 33.84 usrkeys = 927093 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.77 avg-zval = 33.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927093 bytes = 3360720 } seq expand size = 6489651 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5111 value = 3.4505 dict = 1.00 all = 3.5710 } Zip/UnZip{ index = 0.0951 value = 0.2898 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 1.478632 GB avg = 0.121 KB (by entry num) total key len = 0.341974 GB avg = 0.028 KB total ukey len = 0.244267 GB avg = 0.020 KB total ukey num = 0.012213374 Billion total entry num = 0.012213374 Billion write speed all = 0.028316964 MB/sec (with seq num) write speed all = 0.026797270 MB/sec (without seq num) 2024/11/23-02:43:56.968356 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241270: 927093 keys, 37585552 bytes, temperature: kWarm 2024/11/23-02:43:56.968418 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301036968385, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241270, "file_size": 37585552, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3739844043, "table_properties": {"data_size": 31374480, "index_size": 1764032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958604, "raw_average_key_size": 28, "raw_value_size": 108259198, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927093, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301033, "oldest_key_time": 0, "file_creation_time": 1732301033, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241270, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:43:57.008435 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241271.sst) /dev/shm/ToplingTemp/Topling-384761-QSjecZ 2024/11/23-02:43:57.821228 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241271.sst): first pass time = 0.81's, 165.138'MB/sec 2024/11/23-02:43:59.121599 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241271.sst): old prealloc_size = 147639500, real_size = 33608928 2024/11/23-02:43:59.122154 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241271.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935042 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0316 GB avg-val = 115.54 avg-zval = 33.77 2024/11/23-02:43:59.122966 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241271.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:44:00.035301 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241271.sst): second pass time = 1.30's, 83.132'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.425'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3365.006'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 293.411'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32204.069'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.132'MB/sec zip pipeline throughput = 83.132'MB/sec entries = 935042 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.54 avg-zval = 33.77 usrkeys = 935042 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.54 avg-zval = 33.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 935042 bytes = 3389536 } seq expand size = 6545294 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.3761 value = 3.4218 dict = 1.00 all = 3.5462 } Zip/UnZip{ index = 0.0964 value = 0.2922 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 1.593214 GB avg = 0.121 KB (by entry num) total key len = 0.368156 GB avg = 0.028 KB total ukey len = 0.262968 GB avg = 0.020 KB total ukey num = 0.013148416 Billion total entry num = 0.013148416 Billion write speed all = 0.030504863 MB/sec (with seq num) write speed all = 0.028868902 MB/sec (without seq num) 2024/11/23-02:44:00.069758 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241271: 935042 keys, 37848478 bytes, temperature: kWarm 2024/11/23-02:44:00.069825 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301040069793, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241271, "file_size": 37848478, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3339428666, "largest_seqno": 3739844895, "table_properties": {"data_size": 31572688, "index_size": 1802304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26181176, "raw_average_key_size": 28, "raw_value_size": 108036620, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935042, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301036, "oldest_key_time": 0, "file_creation_time": 1732301036, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241271, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:44:00.104465 140025104320064 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241272.sst) /dev/shm/ToplingTemp/Topling-384761-ifxdp0 2024/11/23-02:44:00.407561 140025104320064 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241272.sst): first pass time = 0.30's, 248.575'MB/sec 2024/11/23-02:44:01.129515 140025104320064 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241272.sst): old prealloc_size = 147639500, real_size = 20240064 2024/11/23-02:44:01.129873 140025104320064 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241272.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 522722 prefix = 4 raw-key = 0.0084 GB zip-key = 0.0010 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0607 GB zip-val = 0.0191 GB avg-val = 116.12 avg-zval = 36.57 2024/11/23-02:44:01.130348 140025104320064 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241272.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.075 GB 2024/11/23-02:44:01.629605 140025104320064 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00/241272.sst): second pass time = 0.72's, 84.151'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.50's, 45.225'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2975.418'MB/sec (index lex order gen) rebuild zvType time = 0.43's, 0.306'MB/sec write SST data time = 0.07's, 311.671'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.608'MB, throughput = 20932.810'MB/sec zip my value time = 0.72's, unzip length = 0.061'GB zip my value throughput = 84.151'MB/sec zip pipeline throughput = 84.151'MB/sec entries = 522722 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 36.57 usrkeys = 522722 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 36.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 522722 bytes = 1894880 } seq expand size = 3659054 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0105 GB value = 0.0607 GB dict = 0.61 MB all = 0.0753 GB } __ZipSize{ index = 0.0010 GB value = 0.0191 GB dict = 0.61 MB all = 0.0226 GB } UnZip/Zip{ index = 10.5035 value = 3.1756 dict = 1.00 all = 3.3311 } Zip/UnZip{ index = 0.0952 value = 0.3149 dict = 1.00 all = 0.3002 } ---------------------------- total value len = 1.657571 GB avg = 0.121 KB (by entry num) total key len = 0.382792 GB avg = 0.028 KB total ukey len = 0.273423 GB avg = 0.020 KB total ukey num = 0.013671138 Billion total entry num = 0.013671138 Billion write speed all = 0.031732648 MB/sec (with seq num) write speed all = 0.030031690 MB/sec (without seq num) 2024/11/23-02:44:01.652080 140025104320064 [compaction_job.cc:1948] [default] [JOB 17633] Generated table #241272: 522722 keys, 22615527 bytes, temperature: kWarm 2024/11/23-02:44:01.652140 140025104320064 EVENT_LOG_v1 {"time_micros": 1732301041652107, "cf_name": "default", "job": 17633, "event": "table_file_creation", "file_number": 241272, "file_size": 22615527, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3739845226, "table_properties": {"data_size": 19113840, "index_size": 995328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 14636216, "raw_average_key_size": 28, "raw_value_size": 60698132, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 522722, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301040, "oldest_key_time": 0, "file_creation_time": 1732301040, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241272, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:44:01.676544 140025104320064 [compaction_job.cc:703] job-17633: subcompact[0], size: 0.536944 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17633/att-00 [241258,241259,241260,241261,241262,241263,241264,241265,241266,241267,241268,241269,241270,241271,241272] 2024/11/23-02:44:01.768439 140025104320064 (Original Log Time 2024/11/23-02:43:15.282703) EVENT_LOG_v1 {"time_micros": 1732300995282663, "job": 17633, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [241192], "files_L3": [241249, 241250, 241157, 241158, 241159, 241120, 241121, 241122, 241123, 241124, 241125, 241126, 241127, 241134, 241135, 241227], "score": 0.992597, "input_data_size": 565484895, "oldest_snapshot_seqno": 3740277357} 2024/11/23-02:44:01.768443 140025104320064 (Original Log Time 2024/11/23-02:44:01.767147) [compaction_job.cc:2022] [default] [JOB 17633] Compacted 1@2 + 16@3 files to L3 => 536944498 bytes 2024/11/23-02:44:01.768444 140025104320064 (Original Log Time 2024/11/23-02:44:01.768374) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 12.2 rd, 11.6 wr, level 3, files in(1, 16) out(15 +0 blob) MB in(27.1, 512.2 +0.0 blob) out(512.1 +0.0 blob), read-write-amplify(38.8) write-amplify(18.9) OK, records in: 13757021, records dropped: 85883 output_compression: Snappy 2024/11/23-02:44:01.768449 140025104320064 (Original Log Time 2024/11/23-02:44:01.768420) EVENT_LOG_v1 {"time_micros": 1732301041768397, "cf": "default", "job": 17633, "event": "compaction_finished", "compaction_time_micros": 46393849, "compaction_time_cpu_micros": 20866099, "output_level": 3, "num_output_files": 15, "total_output_size": 536944498, "num_input_records": 13757021, "num_output_records": 13671138, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 335357583, "file_range_sync_nanos": 0, "file_fsync_nanos": 14190, "file_prepare_write_nanos": 33952, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-02:44:01.911097 140025104320064 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732300995,"executesMs":46630,"compactionJobId":17633,"attempt":0,"compactionInputRawBytes":1966578983,"compactionInputZipBytes":565484895,"compactionOutputRawBytes":0,"compactionOutputZipBytes":536944498,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-af859dc89ff5c17a48e67652e8526d69-f27c9a1c734cf2f4-00"}}, response = 2024/11/23-02:44:01.911174 140025104320064 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-17633/att-00: olev 3, work 46.630 s, result 4.952 ms, install 1.321 ms, input{raw 1.832 GiB zip 539.288 MiB}