2024/11/24-01:31:36.213187 139957198378560 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00: fork to child time = 0.007767 sec 2024/11/24-01:31:36.214002 139957198378560 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00 2024/11/24-01:31:36.219905 139957198378560 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00 2024/11/24-01:31:36.220792 139957198378560 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-01:31:36.221132 139957198378560 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-01:31:36.280741 139957198378560 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39395/att-00/MANIFEST-206347 2024/11/24-01:31:36.306065 139957198378560 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39395/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 287357, last_sequence is 4373915689, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 287193 2024/11/24-01:31:36.306073 139957198378560 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-01:31:36.306197 139957198378560 [version_set.cc:5721] Creating manifest 287357 2024/11/24-01:31:36.345011 139957198378560 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-01:31:36.368383 139957198378560 [compaction_job.cc:2437] [default] [JOB 39395] Compacting 1@2 + 16@3 files to L3, score 0.51, subcompactions 1 : 1 2024/11/24-01:31:36.368397 139957198378560 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 29518 Base level 2, inputs: [287314(27MB)], [287335(34MB) 287336(918KB) 287280(34MB) 287281(34MB) 287282(34MB) 287283(35MB) 287284(35MB) 287285(35MB) 287286(6158KB) 287259(30MB) 287260(33MB) 287261(34MB) 287262(32MB) 287263(34MB) 287264(35MB) 287348(33MB)] 2024/11/24-01:31:36.375035 139957198378560 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:31:36.375064 139957198378560 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:31:36.375083 139957198378560 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-01:31:36.375089 139957198378560 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:31:36.375101 139957198378560 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-01:31:37.151883 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287358.sst) /dev/shm/ToplingTemp/Topling-238340-gLjUUj 2024/11/24-01:31:37.815663 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287358.sst): first pass time = 0.66's, 202.215'MB/sec 2024/11/24-01:31:39.139294 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287358.sst): old prealloc_size = 147639500, real_size = 31948576 2024/11/24-01:31:39.139841 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287358.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925857 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0299 GB avg-val = 116.97 avg-zval = 32.34 2024/11/24-01:31:39.140662 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287358.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:31:39.968339 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287358.sst): second pass time = 1.32's, 81.877'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.338'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3307.365'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 286.793'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34985.820'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.877'MB/sec zip pipeline throughput = 81.877'MB/sec entries = 925857 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.97 avg-zval = 32.34 usrkeys = 925857 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.97 avg-zval = 32.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925857 bytes = 3124784 } seq expand size = 6480999 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.4429 value = 3.6166 dict = 1.00 all = 3.7357 } Zip/UnZip{ index = 0.0958 value = 0.2765 dict = 1.00 all = 0.2677 } ---------------------------- total value len = 8.963395 GB avg = 0.168 KB (by entry num) total key len = 1.466745 GB avg = 0.028 KB total ukey len = 1.016395 GB avg = 0.015 KB total ukey num = 0.066476292 Billion total entry num = 0.053284021 Billion write speed all = 0.174452777 MB/sec (with seq num) write speed all = 0.167323020 MB/sec (without seq num) 2024/11/24-01:31:40.013287 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287358: 925857 keys, 35928338 bytes, temperature: kWarm 2024/11/24-01:31:40.013386 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383100013326, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287358, "file_size": 35928338, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4373702275, "table_properties": {"data_size": 29943760, "index_size": 1773184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923996, "raw_average_key_size": 28, "raw_value_size": 108293778, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925857, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383097, "oldest_key_time": 0, "file_creation_time": 1732383097, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287358, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:31:40.054451 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287359.sst) /dev/shm/ToplingTemp/Topling-238340-R28Nkm 2024/11/24-01:31:41.103735 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287359.sst): first pass time = 1.05's, 127.918'MB/sec 2024/11/24-01:31:42.436454 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287359.sst): old prealloc_size = 147639500, real_size = 32076784 2024/11/24-01:31:42.437000 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287359.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925822 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0301 GB avg-val = 116.97 avg-zval = 32.49 2024/11/24-01:31:42.437760 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287359.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:31:43.270735 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287359.sst): second pass time = 1.33's, 81.311'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.220'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3386.245'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 284.027'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35391.524'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.311'MB/sec zip pipeline throughput = 81.311'MB/sec entries = 925822 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 32.49 usrkeys = 925822 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 32.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925822 bytes = 3124672 } seq expand size = 6480754 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5138 value = 3.5997 dict = 1.00 all = 3.7224 } Zip/UnZip{ index = 0.0951 value = 0.2778 dict = 1.00 all = 0.2686 } ---------------------------- total value len = 9.078170 GB avg = 0.167 KB (by entry num) total key len = 1.492668 GB avg = 0.028 KB total ukey len = 1.034912 GB avg = 0.015 KB total ukey num = 0.067402114 Billion total entry num = 0.054209843 Billion write speed all = 0.176796312 MB/sec (with seq num) write speed all = 0.169543075 MB/sec (without seq num) 2024/11/24-01:31:43.316066 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287359: 925822 keys, 36056462 bytes, temperature: kWarm 2024/11/24-01:31:43.316134 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383103316099, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287359, "file_size": 36056462, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4373701940, "table_properties": {"data_size": 30084000, "index_size": 1761152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923016, "raw_average_key_size": 28, "raw_value_size": 108294812, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925822, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383100, "oldest_key_time": 0, "file_creation_time": 1732383100, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287359, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:31:43.353648 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287360.sst) /dev/shm/ToplingTemp/Topling-238340-HADPsm 2024/11/24-01:31:44.483481 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287360.sst): first pass time = 1.13's, 118.798'MB/sec 2024/11/24-01:31:45.804578 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287360.sst): old prealloc_size = 147639500, real_size = 32223984 2024/11/24-01:31:45.805124 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287360.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926100 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0302 GB avg-val = 116.93 avg-zval = 32.63 2024/11/24-01:31:45.805915 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287360.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:31:46.645796 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287360.sst): second pass time = 1.32's, 82.031'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.040'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3395.810'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 287.275'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34476.631'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.031'MB/sec zip pipeline throughput = 82.031'MB/sec entries = 926100 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.93 avg-zval = 32.63 usrkeys = 926100 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.93 avg-zval = 32.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926100 bytes = 3125600 } seq expand size = 6482700 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4426 value = 3.5835 dict = 1.00 all = 3.7072 } Zip/UnZip{ index = 0.0958 value = 0.2791 dict = 1.00 all = 0.2697 } ---------------------------- total value len = 9.192940 GB avg = 0.167 KB (by entry num) total key len = 1.518599 GB avg = 0.028 KB total ukey len = 1.053434 GB avg = 0.015 KB total ukey num = 0.068328214 Billion total entry num = 0.055135943 Billion write speed all = 0.179139404 MB/sec (with seq num) write speed all = 0.171762672 MB/sec (without seq num) 2024/11/24-01:31:46.691029 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287360: 926100 keys, 36204590 bytes, temperature: kWarm 2024/11/24-01:31:46.691097 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383106691061, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287360, "file_size": 36204590, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4373701951, "table_properties": {"data_size": 30218592, "index_size": 1773696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25930800, "raw_average_key_size": 28, "raw_value_size": 108287056, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926100, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383103, "oldest_key_time": 0, "file_creation_time": 1732383103, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287360, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:31:46.728903 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287361.sst) /dev/shm/ToplingTemp/Topling-238340-kW02nk 2024/11/24-01:31:47.869475 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287361.sst): first pass time = 1.14's, 117.679'MB/sec 2024/11/24-01:31:49.216095 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287361.sst): old prealloc_size = 147639500, real_size = 32261680 2024/11/24-01:31:49.216612 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287361.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925895 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0303 GB avg-val = 116.96 avg-zval = 32.69 2024/11/24-01:31:49.217384 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287361.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:31:50.054256 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287361.sst): second pass time = 1.35's, 80.459'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.241'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3541.612'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 283.392'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35102.154'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.459'MB/sec zip pipeline throughput = 80.459'MB/sec entries = 925895 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 32.69 usrkeys = 925895 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 32.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925895 bytes = 3124912 } seq expand size = 6481265 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5150 value = 3.5777 dict = 1.00 all = 3.7034 } Zip/UnZip{ index = 0.0951 value = 0.2795 dict = 1.00 all = 0.2700 } ---------------------------- total value len = 9.307714 GB avg = 0.166 KB (by entry num) total key len = 1.544524 GB avg = 0.028 KB total ukey len = 1.071951 GB avg = 0.015 KB total ukey num = 0.069254109 Billion total entry num = 0.056061838 Billion write speed all = 0.181482106 MB/sec (with seq num) write speed all = 0.173981924 MB/sec (without seq num) 2024/11/24-01:31:50.099306 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287361: 925895 keys, 36241666 bytes, temperature: kWarm 2024/11/24-01:31:50.099380 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383110099341, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287361, "file_size": 36241666, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4373702441, "table_properties": {"data_size": 30268960, "index_size": 1761088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25925060, "raw_average_key_size": 28, "raw_value_size": 108292762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925895, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383106, "oldest_key_time": 0, "file_creation_time": 1732383106, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287361, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:31:50.136683 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287362.sst) /dev/shm/ToplingTemp/Topling-238340-e64hMm 2024/11/24-01:31:51.132646 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287362.sst): first pass time = 1.00's, 134.767'MB/sec 2024/11/24-01:31:52.473611 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287362.sst): old prealloc_size = 147639500, real_size = 32593840 2024/11/24-01:31:52.474179 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287362.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925892 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 116.96 avg-zval = 33.05 2024/11/24-01:31:52.474918 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287362.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:31:53.330444 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287362.sst): second pass time = 1.34's, 80.797'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.686'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3180.301'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 290.459'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 29142.233'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.797'MB/sec zip pipeline throughput = 80.797'MB/sec entries = 925892 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 33.05 usrkeys = 925892 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 33.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925892 bytes = 3124896 } seq expand size = 6481244 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5116 value = 3.5389 dict = 1.00 all = 3.6698 } Zip/UnZip{ index = 0.0951 value = 0.2826 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 9.422488 GB avg = 0.165 KB (by entry num) total key len = 1.570449 GB avg = 0.028 KB total ukey len = 1.090469 GB avg = 0.016 KB total ukey num = 0.070180001 Billion total entry num = 0.056987730 Billion write speed all = 0.183824947 MB/sec (with seq num) write speed all = 0.176201312 MB/sec (without seq num) 2024/11/24-01:31:53.375151 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287362: 925892 keys, 36573730 bytes, temperature: kWarm 2024/11/24-01:31:53.375221 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383113375186, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287362, "file_size": 36573730, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4373701986, "table_properties": {"data_size": 30600544, "index_size": 1761664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924976, "raw_average_key_size": 28, "raw_value_size": 108292828, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925892, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383110, "oldest_key_time": 0, "file_creation_time": 1732383110, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287362, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:31:53.411328 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287363.sst) /dev/shm/ToplingTemp/Topling-238340-ngaM3m 2024/11/24-01:31:54.439738 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287363.sst): first pass time = 1.03's, 130.514'MB/sec 2024/11/24-01:31:55.806092 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287363.sst): old prealloc_size = 147639500, real_size = 32890064 2024/11/24-01:31:55.806624 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287363.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925726 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.99 avg-zval = 33.38 2024/11/24-01:31:55.807392 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287363.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:31:56.651779 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287363.sst): second pass time = 1.37's, 79.308'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.598'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3462.541'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 289.830'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27931.275'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.308'MB/sec zip pipeline throughput = 79.308'MB/sec entries = 925726 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.99 avg-zval = 33.38 usrkeys = 925726 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.99 avg-zval = 33.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925726 bytes = 3124336 } seq expand size = 6480082 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5143 value = 3.5051 dict = 1.00 all = 3.6404 } Zip/UnZip{ index = 0.0951 value = 0.2853 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 9.537266 GB avg = 0.165 KB (by entry num) total key len = 1.596369 GB avg = 0.028 KB total ukey len = 1.108984 GB avg = 0.016 KB total ukey num = 0.071105727 Billion total entry num = 0.057913456 Billion write speed all = 0.186167371 MB/sec (with seq num) write speed all = 0.178420326 MB/sec (without seq num) 2024/11/24-01:31:56.697798 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287363: 925726 keys, 36869022 bytes, temperature: kWarm 2024/11/24-01:31:56.697870 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383116697834, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287363, "file_size": 36869022, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4373702606, "table_properties": {"data_size": 30897536, "index_size": 1760896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920328, "raw_average_key_size": 28, "raw_value_size": 108297500, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925726, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383113, "oldest_key_time": 0, "file_creation_time": 1732383113, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287363, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:31:56.741679 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287364.sst) /dev/shm/ToplingTemp/Topling-238340-qI0qAm 2024/11/24-01:31:57.782863 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287364.sst): first pass time = 1.04's, 128.916'MB/sec 2024/11/24-01:31:59.110108 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287364.sst): old prealloc_size = 147639500, real_size = 33204784 2024/11/24-01:31:59.110648 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287364.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925990 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.95 avg-zval = 33.69 2024/11/24-01:31:59.111405 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287364.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:31:59.948048 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287364.sst): second pass time = 1.33's, 81.650'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.382'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3433.267'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 295.922'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36583.345'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.650'MB/sec zip pipeline throughput = 81.650'MB/sec entries = 925990 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.95 avg-zval = 33.69 usrkeys = 925990 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.95 avg-zval = 33.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925990 bytes = 3125232 } seq expand size = 6481930 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4451 value = 3.4708 dict = 1.00 all = 3.6091 } Zip/UnZip{ index = 0.0957 value = 0.2881 dict = 1.00 all = 0.2771 } ---------------------------- total value len = 9.652038 GB avg = 0.164 KB (by entry num) total key len = 1.622297 GB avg = 0.028 KB total ukey len = 1.127504 GB avg = 0.016 KB total ukey num = 0.072031717 Billion total entry num = 0.058839446 Billion write speed all = 0.188509643 MB/sec (with seq num) write speed all = 0.180639163 MB/sec (without seq num) 2024/11/24-01:31:59.994102 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287364: 925990 keys, 37188720 bytes, temperature: kWarm 2024/11/24-01:31:59.994174 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383119994140, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287364, "file_size": 37188720, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4373702025, "table_properties": {"data_size": 31200032, "index_size": 1773056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25927720, "raw_average_key_size": 28, "raw_value_size": 108290008, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925990, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383116, "oldest_key_time": 0, "file_creation_time": 1732383116, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287364, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:32:00.043156 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287365.sst) /dev/shm/ToplingTemp/Topling-238340-aXTOZj 2024/11/24-01:32:01.146439 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287365.sst): first pass time = 1.10's, 121.660'MB/sec 2024/11/24-01:32:02.432721 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287365.sst): old prealloc_size = 147639500, real_size = 28705952 2024/11/24-01:32:02.433287 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287365.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 940639 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1079 GB zip-val = 0.0267 GB avg-val = 114.69 avg-zval = 28.34 2024/11/24-01:32:02.434702 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287365.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:32:03.265928 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287365.sst): second pass time = 1.28's, 83.957'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 39.278'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3317.207'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 255.733'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.076'MB, throughput = 25487.889'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 83.957'MB/sec zip pipeline throughput = 85.835'MB/sec entries = 940639 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.69 avg-zval = 28.34 usrkeys = 940639 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.69 avg-zval = 28.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 940639 bytes = 3174672 } seq expand size = 6584473 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0267 GB dict = 1.08 MB all = 0.0327 GB } UnZip/Zip{ index = 10.3763 value = 4.0469 dict = 1.00 all = 4.1013 } Zip/UnZip{ index = 0.0964 value = 0.2471 dict = 1.00 all = 0.2438 } ---------------------------- total value len = 9.766502 GB avg = 0.163 KB (by entry num) total key len = 1.648635 GB avg = 0.028 KB total ukey len = 1.146316 GB avg = 0.016 KB total ukey num = 0.072972356 Billion total entry num = 0.059780085 Billion write speed all = 0.190853304 MB/sec (with seq num) write speed all = 0.182857446 MB/sec (without seq num) 2024/11/24-01:32:03.308840 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287365: 940639 keys, 32725790 bytes, temperature: kWarm 2024/11/24-01:32:03.308908 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383123308871, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287365, "file_size": 32725790, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4373701170, "table_properties": {"data_size": 26657552, "index_size": 1813056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26337892, "raw_average_key_size": 28, "raw_value_size": 107879906, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 940639, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383120, "oldest_key_time": 0, "file_creation_time": 1732383120, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287365, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:32:03.357112 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287366.sst) /dev/shm/ToplingTemp/Topling-238340-B6eHPk 2024/11/24-01:32:04.324186 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287366.sst): first pass time = 0.97's, 138.793'MB/sec 2024/11/24-01:32:05.673657 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287366.sst): old prealloc_size = 147639500, real_size = 33080576 2024/11/24-01:32:05.674252 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287366.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925874 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.96 avg-zval = 33.58 2024/11/24-01:32:05.675584 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287366.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:32:06.522771 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287366.sst): second pass time = 1.35's, 80.296'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.647'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3092.087'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 288.688'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35115.452'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.296'MB/sec zip pipeline throughput = 80.296'MB/sec entries = 925874 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.96 avg-zval = 33.58 usrkeys = 925874 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.96 avg-zval = 33.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925874 bytes = 3124832 } seq expand size = 6481118 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5114 value = 3.4835 dict = 1.00 all = 3.6216 } Zip/UnZip{ index = 0.0951 value = 0.2871 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 9.881276 GB avg = 0.163 KB (by entry num) total key len = 1.674559 GB avg = 0.028 KB total ukey len = 1.164834 GB avg = 0.016 KB total ukey num = 0.073898230 Billion total entry num = 0.060705959 Billion write speed all = 0.193195175 MB/sec (with seq num) write speed all = 0.185075920 MB/sec (without seq num) 2024/11/24-01:32:06.569054 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287366: 925874 keys, 37060154 bytes, temperature: kWarm 2024/11/24-01:32:06.569124 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383126569088, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287366, "file_size": 37060154, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4373702572, "table_properties": {"data_size": 31087280, "index_size": 1761664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25924472, "raw_average_key_size": 28, "raw_value_size": 108293400, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925874, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383123, "oldest_key_time": 0, "file_creation_time": 1732383123, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287366, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:32:06.618749 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287367.sst) /dev/shm/ToplingTemp/Topling-238340-hrj6Zm 2024/11/24-01:32:07.537991 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287367.sst): first pass time = 0.92's, 146.017'MB/sec 2024/11/24-01:32:08.809902 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287367.sst): old prealloc_size = 147639500, real_size = 28980832 2024/11/24-01:32:08.810490 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287367.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 943041 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1078 GB zip-val = 0.0270 GB avg-val = 114.32 avg-zval = 28.58 2024/11/24-01:32:08.811889 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287367.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:32:09.648435 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287367.sst): second pass time = 1.27's, 84.814'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 39.365'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3265.836'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.332'MB/sec write SST data time = 0.13's, 256.134'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.075'MB, throughput = 31359.111'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.814'MB/sec zip pipeline throughput = 84.814'MB/sec entries = 943041 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.32 avg-zval = 28.58 usrkeys = 943041 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.32 avg-zval = 28.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 943041 bytes = 3182784 } seq expand size = 6601287 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0270 GB dict = 1.07 MB all = 0.0330 GB } UnZip/Zip{ index = 10.5130 value = 4.0003 dict = 1.00 all = 4.0662 } Zip/UnZip{ index = 0.0951 value = 0.2500 dict = 1.00 all = 0.2459 } ---------------------------- total value len = 9.995690 GB avg = 0.162 KB (by entry num) total key len = 1.700964 GB avg = 0.028 KB total ukey len = 1.183695 GB avg = 0.016 KB total ukey num = 0.074841271 Billion total entry num = 0.061649000 Billion write speed all = 0.195539228 MB/sec (with seq num) write speed all = 0.187294274 MB/sec (without seq num) 2024/11/24-01:32:09.689356 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287367: 943041 keys, 33008390 bytes, temperature: kWarm 2024/11/24-01:32:09.689427 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383129689390, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287367, "file_size": 33008390, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4373702809, "table_properties": {"data_size": 26950864, "index_size": 1794048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26405148, "raw_average_key_size": 28, "raw_value_size": 107812630, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 943041, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383126, "oldest_key_time": 0, "file_creation_time": 1732383126, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287367, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:32:09.729605 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287368.sst) /dev/shm/ToplingTemp/Topling-238340-qmQOPm 2024/11/24-01:32:10.711255 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287368.sst): first pass time = 0.98's, 136.732'MB/sec 2024/11/24-01:32:11.997276 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287368.sst): old prealloc_size = 147639500, real_size = 31139040 2024/11/24-01:32:11.997846 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287368.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922654 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.47 avg-zval = 31.57 2024/11/24-01:32:11.999278 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287368.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:32:12.810281 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287368.sst): second pass time = 1.29's, 84.331'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.186'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3246.910'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 275.343'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31923.834'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.331'MB/sec zip pipeline throughput = 84.331'MB/sec entries = 922654 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.47 avg-zval = 31.57 usrkeys = 922654 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.47 avg-zval = 31.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922654 bytes = 3113968 } seq expand size = 6458578 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.3790 value = 3.7206 dict = 1.00 all = 3.8228 } Zip/UnZip{ index = 0.0963 value = 0.2688 dict = 1.00 all = 0.2616 } ---------------------------- total value len = 10.110532 GB avg = 0.162 KB (by entry num) total key len = 1.726799 GB avg = 0.028 KB total ukey len = 1.202148 GB avg = 0.016 KB total ukey num = 0.075763925 Billion total entry num = 0.062571654 Billion write speed all = 0.197880529 MB/sec (with seq num) write speed all = 0.189512622 MB/sec (without seq num) 2024/11/24-01:32:12.854750 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287368: 922654 keys, 35109432 bytes, temperature: kWarm 2024/11/24-01:32:12.854822 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383132854786, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287368, "file_size": 35109432, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4272725028, "largest_seqno": 4373702823, "table_properties": {"data_size": 29130256, "index_size": 1777920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25834312, "raw_average_key_size": 28, "raw_value_size": 108383432, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922654, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383129, "oldest_key_time": 0, "file_creation_time": 1732383129, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287368, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:32:12.896882 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287369.sst) /dev/shm/ToplingTemp/Topling-238340-zO8h2l 2024/11/24-01:32:13.761886 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287369.sst): first pass time = 0.86's, 155.173'MB/sec 2024/11/24-01:32:15.078043 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287369.sst): old prealloc_size = 147639500, real_size = 32915552 2024/11/24-01:32:15.078572 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287369.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929526 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.39 avg-zval = 33.26 2024/11/24-01:32:15.080611 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287369.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:32:15.940627 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287369.sst): second pass time = 1.31's, 82.276'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.786'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3498.751'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 285.736'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33994.960'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.276'MB/sec zip pipeline throughput = 82.276'MB/sec entries = 929526 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.39 avg-zval = 33.26 usrkeys = 929526 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.39 avg-zval = 33.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929526 bytes = 3137168 } seq expand size = 6506682 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5127 value = 3.4997 dict = 1.00 all = 3.6368 } Zip/UnZip{ index = 0.0951 value = 0.2857 dict = 1.00 all = 0.2750 } ---------------------------- total value len = 10.225230 GB avg = 0.161 KB (by entry num) total key len = 1.752826 GB avg = 0.028 KB total ukey len = 1.220738 GB avg = 0.016 KB total ukey num = 0.076693451 Billion total entry num = 0.063501180 Billion write speed all = 0.200222494 MB/sec (with seq num) write speed all = 0.191730722 MB/sec (without seq num) 2024/11/24-01:32:15.985917 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287369: 929526 keys, 36905400 bytes, temperature: kWarm 2024/11/24-01:32:15.986012 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383135985952, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287369, "file_size": 36905400, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4373702962, "table_properties": {"data_size": 30914576, "index_size": 1768384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26026728, "raw_average_key_size": 28, "raw_value_size": 108191044, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929526, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383132, "oldest_key_time": 0, "file_creation_time": 1732383132, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287369, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:32:16.022617 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287370.sst) /dev/shm/ToplingTemp/Topling-238340-wjrenm 2024/11/24-01:32:17.024808 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287370.sst): first pass time = 1.00's, 133.929'MB/sec 2024/11/24-01:32:18.313972 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287370.sst): old prealloc_size = 147639500, real_size = 32060480 2024/11/24-01:32:18.314503 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287370.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930166 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0301 GB avg-val = 116.29 avg-zval = 32.31 2024/11/24-01:32:18.315251 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287370.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:32:19.158199 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287370.sst): second pass time = 1.29's, 83.974'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.705'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3605.435'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 279.597'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32894.106'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.974'MB/sec zip pipeline throughput = 83.974'MB/sec entries = 930166 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.29 avg-zval = 32.31 usrkeys = 930166 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.29 avg-zval = 32.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930166 bytes = 3139328 } seq expand size = 6511162 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5120 value = 3.5988 dict = 1.00 all = 3.7229 } Zip/UnZip{ index = 0.0951 value = 0.2779 dict = 1.00 all = 0.2686 } ---------------------------- total value len = 10.339914 GB avg = 0.160 KB (by entry num) total key len = 1.778870 GB avg = 0.028 KB total ukey len = 1.239342 GB avg = 0.016 KB total ukey num = 0.077623617 Billion total entry num = 0.064431346 Billion write speed all = 0.202563992 MB/sec (with seq num) write speed all = 0.193948296 MB/sec (without seq num) 2024/11/24-01:32:19.203738 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287370: 930166 keys, 36051964 bytes, temperature: kWarm 2024/11/24-01:32:19.203810 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383139203774, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287370, "file_size": 36051964, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4373702998, "table_properties": {"data_size": 30058032, "index_size": 1769728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26044648, "raw_average_key_size": 28, "raw_value_size": 108173144, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930166, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383136, "oldest_key_time": 0, "file_creation_time": 1732383136, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287370, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:32:19.244358 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287371.sst) /dev/shm/ToplingTemp/Topling-238340-updRCn 2024/11/24-01:32:20.243430 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287371.sst): first pass time = 1.00's, 134.348'MB/sec 2024/11/24-01:32:21.535792 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287371.sst): old prealloc_size = 147639500, real_size = 30949744 2024/11/24-01:32:21.536367 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287371.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924288 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.21 avg-zval = 31.31 2024/11/24-01:32:21.537124 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287371.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-01:32:22.351919 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287371.sst): second pass time = 1.29's, 83.919'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.794'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3242.632'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 273.484'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35353.810'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.919'MB/sec zip pipeline throughput = 83.919'MB/sec entries = 924288 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.21 avg-zval = 31.31 usrkeys = 924288 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.21 avg-zval = 31.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924288 bytes = 3119488 } seq expand size = 6470016 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.3784 value = 3.7439 dict = 1.00 all = 3.8431 } Zip/UnZip{ index = 0.0964 value = 0.2671 dict = 1.00 all = 0.2602 } ---------------------------- total value len = 10.454722 GB avg = 0.160 KB (by entry num) total key len = 1.804750 GB avg = 0.028 KB total ukey len = 1.257827 GB avg = 0.016 KB total ukey num = 0.078547905 Billion total entry num = 0.065355634 Billion write speed all = 0.204904633 MB/sec (with seq num) write speed all = 0.196165808 MB/sec (without seq num) 2024/11/24-01:32:22.394292 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287371: 924288 keys, 34924160 bytes, temperature: kWarm 2024/11/24-01:32:22.394365 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383142394329, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287371, "file_size": 34924160, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4285769799, "largest_seqno": 4373702843, "table_properties": {"data_size": 28937312, "index_size": 1781184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25880064, "raw_average_key_size": 28, "raw_value_size": 108337692, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924288, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383139, "oldest_key_time": 0, "file_creation_time": 1732383139, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287371, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:32:22.433342 139957198378560 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287372.sst) /dev/shm/ToplingTemp/Topling-238340-MeA2Kk 2024/11/24-01:32:22.608038 139957198378560 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287372.sst): first pass time = 0.17's, 239.322'MB/sec 2024/11/24-01:32:23.023074 139957198378560 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287372.sst): old prealloc_size = 147639500, real_size = 11462912 2024/11/24-01:32:23.023287 139957198378560 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287372.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 291406 prefix = 4 raw-key = 0.0047 GB zip-key = 0.0006 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0336 GB zip-val = 0.0108 GB avg-val = 115.44 avg-zval = 37.18 2024/11/24-01:32:23.023641 139957198378560 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287372.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0007 GB, waited 0.000 sec, Key+Value = 0.042 GB 2024/11/24-01:32:23.281795 139957198378560 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00/287372.sst): second pass time = 0.41's, 81.191'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.26's, 49.161'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2900.688'MB/sec (index lex order gen) rebuild zvType time = 0.22's, 0.337'MB/sec write SST data time = 0.04's, 301.233'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.341'MB, throughput = 10600.348'MB/sec zip my value time = 0.41's, unzip length = 0.034'GB zip my value throughput = 81.191'MB/sec zip pipeline throughput = 81.191'MB/sec entries = 291406 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.44 avg-zval = 37.18 usrkeys = 291406 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.44 avg-zval = 37.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 291406 bytes = 983504 } seq expand size = 2039842 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0058 GB value = 0.0336 GB dict = 0.34 MB all = 0.0418 GB } __ZipSize{ index = 0.0006 GB value = 0.0108 GB dict = 0.34 MB all = 0.0127 GB } UnZip/Zip{ index = 10.4949 value = 3.1048 dict = 1.00 all = 3.2868 } Zip/UnZip{ index = 0.0953 value = 0.3221 dict = 1.00 all = 0.3043 } ---------------------------- total value len = 10.490401 GB avg = 0.160 KB (by entry num) total key len = 1.812910 GB avg = 0.028 KB total ukey len = 1.263656 GB avg = 0.016 KB total ukey num = 0.078839311 Billion total entry num = 0.065647040 Billion write speed all = 0.205634149 MB/sec (with seq num) write speed all = 0.196856496 MB/sec (without seq num) 2024/11/24-01:32:23.303037 139957198378560 [compaction_job.cc:1948] [default] [JOB 39395] Generated table #287372: 291406 keys, 12717199 bytes, temperature: kWarm 2024/11/24-01:32:23.303106 139957198378560 EVENT_LOG_v1 {"time_micros": 1732383143303071, "cf_name": "default", "job": 39395, "event": "table_file_creation", "file_number": 287372, "file_size": 12717199, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4373702652, "table_properties": {"data_size": 10834544, "index_size": 555328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 8159368, "raw_average_key_size": 28, "raw_value_size": 33639028, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 291406, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732383142, "oldest_key_time": 0, "file_creation_time": 1732383142, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 287372, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-01:32:23.317714 139957198378560 [compaction_job.cc:703] job-39395: subcompact[0], size: 0.513565 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-39395/att-00 [287358,287359,287360,287361,287362,287363,287364,287365,287366,287367,287368,287369,287370,287371,287372] 2024/11/24-01:32:23.405766 139957198378560 (Original Log Time 2024/11/24-01:31:36.368455) EVENT_LOG_v1 {"time_micros": 1732383096368413, "job": 39395, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [287314], "files_L3": [287335, 287336, 287280, 287281, 287282, 287283, 287284, 287285, 287286, 287259, 287260, 287261, 287262, 287263, 287264, 287348], "score": 0.513163, "input_data_size": 538250692, "oldest_snapshot_seqno": 4373915689} 2024/11/24-01:32:23.405770 139957198378560 (Original Log Time 2024/11/24-01:32:23.404519) [compaction_job.cc:2022] [default] [JOB 39395] Compacted 1@2 + 16@3 files to L3 => 513565017 bytes 2024/11/24-01:32:23.405772 139957198378560 (Original Log Time 2024/11/24-01:32:23.405700) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 11.5 rd, 10.9 wr, level 3, files in(1, 16) out(15 +0 blob) MB in(27.1, 486.2 +0.0 blob) out(489.8 +0.0 blob), read-write-amplify(37.0) write-amplify(18.1) OK, records in: 13389065, records dropped: 100189 output_compression: Snappy 2024/11/24-01:32:23.405778 139957198378560 (Original Log Time 2024/11/24-01:32:23.405745) EVENT_LOG_v1 {"time_micros": 1732383143405723, "cf": "default", "job": 39395, "event": "compaction_finished", "compaction_time_micros": 46949267, "compaction_time_cpu_micros": 20472577, "output_level": 3, "num_output_files": 15, "total_output_size": 513565017, "num_input_records": 13389065, "num_output_records": 13288876, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 350166917, "file_range_sync_nanos": 0, "file_fsync_nanos": 11584, "file_prepare_write_nanos": 53422, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/24-01:32:23.551139 139957198378560 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732383096,"executesMs":47183,"compactionJobId":39395,"attempt":0,"compactionInputRawBytes":1935074386,"compactionInputZipBytes":538250692,"compactionOutputRawBytes":0,"compactionOutputZipBytes":513565017,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-1b8298e1bda6fa1e8ccfd305c3f5d07b-b1e1e10556794cc0-00"}}, response = 2024/11/24-01:32:23.551248 139957198378560 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-39395/att-00: olev 3, work 47.183 s, result 7.665 ms, install 1.283 ms, input{raw 1.802 GiB zip 513.316 MiB}