2024/11/23-20:03:35.252562 139957584442944 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00: fork to child time = 0.007501 sec 2024/11/23-20:03:35.253318 139957584442944 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00 2024/11/23-20:03:35.260456 139957584442944 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00 2024/11/23-20:03:35.261364 139957584442944 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-20:03:35.261615 139957584442944 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-20:03:35.313923 139957584442944 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34781/att-00/MANIFEST-206347 2024/11/23-20:03:35.360917 139957584442944 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34781/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 278468, last_sequence is 4233070807, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 278281 2024/11/23-20:03:35.360925 139957584442944 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-20:03:35.361133 139957584442944 [version_set.cc:5721] Creating manifest 278468 2024/11/23-20:03:35.395611 139957584442944 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-20:03:35.420931 139957584442944 [compaction_job.cc:2437] [default] [JOB 34781] Compacting 1@2 + 18@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-20:03:35.420948 139957584442944 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 26034 Base level 2, inputs: [278408(27MB)], [278435(36MB) 278436(2926KB) 278363(38MB) 278364(37MB) 278365(37MB) 278366(38MB) 278367(35MB) 278368(34MB) 278369(37MB) 278370(33MB) 278371(37MB) 278372(35MB) 278373(34MB) 278374(36MB) 278375(35MB) 278376(36MB) 278377(37MB) 278437(36MB)] 2024/11/23-20:03:35.429937 139957584442944 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:03:35.429961 139957584442944 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:03:35.429989 139957584442944 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:03:35.429995 139957584442944 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-20:03:35.430005 139957584442944 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-20:03:35.975224 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278469.sst) /dev/shm/ToplingTemp/Topling-174138-9giDZF 2024/11/23-20:03:36.534353 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278469.sst): first pass time = 0.56's, 240.071'MB/sec 2024/11/23-20:03:38.198294 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278469.sst): old prealloc_size = 147639500, real_size = 34396944 2024/11/23-20:03:38.198866 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278469.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930175 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.29 avg-zval = 34.83 2024/11/23-20:03:38.199628 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278469.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:39.243479 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278469.sst): second pass time = 1.66's, 65.043'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 36.618'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3221.579'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.253'MB/sec write SST data time = 0.12's, 307.242'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33099.991'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.043'MB/sec zip pipeline throughput = 65.564'MB/sec entries = 930175 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.29 avg-zval = 34.83 usrkeys = 930175 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.29 avg-zval = 34.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930175 bytes = 3023088 } seq expand size = 6511225 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.5132 value = 3.3392 dict = 1.00 all = 3.5069 } Zip/UnZip{ index = 0.0951 value = 0.2995 dict = 1.00 all = 0.2851 } ---------------------------- total value len = 8.963304 GB avg = 0.168 KB (by entry num) total key len = 1.466866 GB avg = 0.028 KB total ukey len = 1.016482 GB avg = 0.015 KB total ukey num = 0.066480610 Billion total entry num = 0.053288339 Billion write speed all = 0.260058403 MB/sec (with seq num) write speed all = 0.249429176 MB/sec (without seq num) 2024/11/23-20:03:39.291389 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278469: 930175 keys, 38272124 bytes, temperature: kWarm 2024/11/23-20:03:39.291485 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363419291429, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278469, "file_size": 38272124, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4191256266, "largest_seqno": 4232676849, "table_properties": {"data_size": 32394688, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26044900, "raw_average_key_size": 28, "raw_value_size": 108172854, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930175, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363415, "oldest_key_time": 0, "file_creation_time": 1732363415, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278469, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:39.328999 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278470.sst) /dev/shm/ToplingTemp/Topling-174138-ByXLFF 2024/11/23-20:03:40.754906 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278470.sst): first pass time = 1.43's, 94.130'MB/sec 2024/11/23-20:03:42.545941 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278470.sst): old prealloc_size = 147639500, real_size = 36636080 2024/11/23-20:03:42.546543 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278470.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933378 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1081 GB zip-val = 0.0346 GB avg-val = 115.80 avg-zval = 37.11 2024/11/23-20:03:42.547369 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278470.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:43.363871 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278470.sst): second pass time = 1.78's, 60.760'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 50.117'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3125.586'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.338'MB/sec write SST data time = 0.13's, 320.910'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 632.245'MB/sec zip my value time = 1.78's, unzip length = 0.108'GB zip my value throughput = 60.760'MB/sec zip pipeline throughput = 105.427'MB/sec entries = 933378 avg-key = 28.00 avg-zkey = 1.89 avg-val = 115.80 avg-zval = 37.11 usrkeys = 933378 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.80 avg-zval = 37.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933378 bytes = 3500176 } seq expand size = 6533646 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0410 GB } UnZip/Zip{ index = 10.5835 value = 3.1203 dict = 1.00 all = 3.2744 } Zip/UnZip{ index = 0.0945 value = 0.3205 dict = 1.00 all = 0.3054 } ---------------------------- total value len = 9.077921 GB avg = 0.167 KB (by entry num) total key len = 1.493001 GB avg = 0.028 KB total ukey len = 1.035149 GB avg = 0.015 KB total ukey num = 0.067413988 Billion total entry num = 0.054221717 Billion write speed all = 0.263540725 MB/sec (with seq num) write speed all = 0.252726432 MB/sec (without seq num) 2024/11/23-20:03:43.415199 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278470: 933378 keys, 40990662 bytes, temperature: kWarm 2024/11/23-20:03:43.415277 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363423415233, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278470, "file_size": 40990662, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3591110068, "largest_seqno": 4232676778, "table_properties": {"data_size": 34638688, "index_size": 1763840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26134584, "raw_average_key_size": 28, "raw_value_size": 108083268, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933378, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363419, "oldest_key_time": 0, "file_creation_time": 1732363419, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278470, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:43.450771 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278471.sst) /dev/shm/ToplingTemp/Topling-174138-EUarcH 2024/11/23-20:03:44.319658 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278471.sst): first pass time = 0.87's, 154.478'MB/sec 2024/11/23-20:03:46.011936 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278471.sst): old prealloc_size = 147639500, real_size = 36803264 2024/11/23-20:03:46.012525 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278471.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931183 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0348 GB avg-val = 116.14 avg-zval = 37.35 2024/11/23-20:03:46.013170 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278471.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:46.790366 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278471.sst): second pass time = 1.69's, 63.934'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 51.571'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3165.874'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.358'MB/sec write SST data time = 0.13's, 313.628'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28048.336'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 63.934'MB/sec zip pipeline throughput = 134.795'MB/sec entries = 931183 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.14 avg-zval = 37.35 usrkeys = 931183 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.83 avg-zval = 37.35 TagRS{ kinds = RS_Key0_TagN bytes = 145600 } TagArray{ size = 624391 bytes = 2341488 } seq expand size = 4370737 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0401 GB } UnZip/Zip{ index = 10.3745 value = 3.1098 dict = 1.00 all = 3.3433 } Zip/UnZip{ index = 0.0964 value = 0.3216 dict = 1.00 all = 0.2991 } ---------------------------- total value len = 9.190436 GB avg = 0.167 KB (by entry num) total key len = 1.519074 GB avg = 0.028 KB total ukey len = 1.053773 GB avg = 0.015 KB total ukey num = 0.068345171 Billion total entry num = 0.055152900 Billion write speed all = 0.266973032 MB/sec (with seq num) write speed all = 0.255973958 MB/sec (without seq num) 2024/11/23-20:03:46.841059 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278471: 931183 keys, 40145238 bytes, temperature: kWarm 2024/11/23-20:03:46.841131 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363426841094, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278471, "file_size": 40145238, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775947, "table_properties": {"data_size": 34775152, "index_size": 1795136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26073124, "raw_average_key_size": 28, "raw_value_size": 108144702, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931183, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363423, "oldest_key_time": 0, "file_creation_time": 1732363423, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278471, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:46.881705 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278472.sst) /dev/shm/ToplingTemp/Topling-174138-IOq6BF 2024/11/23-20:03:47.891464 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278472.sst): first pass time = 1.01's, 132.925'MB/sec 2024/11/23-20:03:49.360401 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278472.sst): old prealloc_size = 147639500, real_size = 35577488 2024/11/23-20:03:49.360941 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278472.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924151 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0336 GB avg-val = 117.23 avg-zval = 36.32 2024/11/23-20:03:49.361753 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278472.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:50.197842 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278472.sst): second pass time = 1.47's, 73.795'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.508'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3441.758'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 316.040'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32834.901'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.795'MB/sec zip pipeline throughput = 95.575'MB/sec entries = 924151 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.23 avg-zval = 36.32 usrkeys = 924151 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.23 avg-zval = 36.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924151 bytes = 3350064 } seq expand size = 6469057 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0336 GB dict = 1.08 MB all = 0.0398 GB } UnZip/Zip{ index = 10.3772 value = 3.2278 dict = 1.00 all = 3.3736 } Zip/UnZip{ index = 0.0964 value = 0.3098 dict = 1.00 all = 0.2964 } ---------------------------- total value len = 9.305247 GB avg = 0.166 KB (by entry num) total key len = 1.544950 GB avg = 0.028 KB total ukey len = 1.072256 GB avg = 0.015 KB total ukey num = 0.069269322 Billion total entry num = 0.056077051 Billion write speed all = 0.270457184 MB/sec (with seq num) write speed all = 0.259274757 MB/sec (without seq num) 2024/11/23-20:03:50.246087 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278472: 924151 keys, 39784198 bytes, temperature: kWarm 2024/11/23-20:03:50.246157 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363430246122, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278472, "file_size": 39784198, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33565120, "index_size": 1781120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876228, "raw_average_key_size": 28, "raw_value_size": 108341550, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924151, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363426, "oldest_key_time": 0, "file_creation_time": 1732363426, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278472, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:50.291918 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278473.sst) /dev/shm/ToplingTemp/Topling-174138-2xEvVF 2024/11/23-20:03:51.151514 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278473.sst): first pass time = 0.86's, 156.147'MB/sec 2024/11/23-20:03:52.517437 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278473.sst): old prealloc_size = 147639500, real_size = 35524544 2024/11/23-20:03:52.518038 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278473.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930745 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.20 avg-zval = 36.00 2024/11/23-20:03:52.518758 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278473.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:53.339751 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278473.sst): second pass time = 1.36's, 79.250'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 47.829'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3089.377'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.339'MB/sec write SST data time = 0.14's, 291.199'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34232.877'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.250'MB/sec zip pipeline throughput = 79.250'MB/sec entries = 930745 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.20 avg-zval = 36.00 usrkeys = 930745 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.83 avg-zval = 36.00 TagRS{ kinds = RS_Key0_TagN bytes = 145520 } TagArray{ size = 747748 bytes = 2804064 } seq expand size = 5234236 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4467 value = 3.2276 dict = 1.00 all = 3.4127 } Zip/UnZip{ index = 0.0957 value = 0.3098 dict = 1.00 all = 0.2930 } ---------------------------- total value len = 9.418638 GB avg = 0.165 KB (by entry num) total key len = 1.571011 GB avg = 0.028 KB total ukey len = 1.090871 GB avg = 0.016 KB total ukey num = 0.070200067 Billion total entry num = 0.057007796 Billion write speed all = 0.273911781 MB/sec (with seq num) write speed all = 0.262544643 MB/sec (without seq num) 2024/11/23-20:03:53.390363 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278473: 930745 keys, 39329312 bytes, temperature: kWarm 2024/11/23-20:03:53.390434 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363433390398, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278473, "file_size": 39329312, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 33509808, "index_size": 1781888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26060860, "raw_average_key_size": 28, "raw_value_size": 108156994, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930745, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363430, "oldest_key_time": 0, "file_creation_time": 1732363430, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278473, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:53.427441 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278474.sst) /dev/shm/ToplingTemp/Topling-174138-5CvwQG 2024/11/23-20:03:54.466838 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278474.sst): first pass time = 1.04's, 129.137'MB/sec 2024/11/23-20:03:55.800775 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278474.sst): old prealloc_size = 147639500, real_size = 32857264 2024/11/23-20:03:55.801382 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278474.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.49 avg-zval = 33.21 2024/11/23-20:03:55.802168 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278474.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:56.654088 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278474.sst): second pass time = 1.33's, 81.168'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.453'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3040.929'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 287.622'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32860.961'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.168'MB/sec zip pipeline throughput = 81.168'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.49 avg-zval = 33.21 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.49 avg-zval = 33.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3367296 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4452 value = 3.5080 dict = 1.00 all = 3.6199 } Zip/UnZip{ index = 0.0957 value = 0.2851 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 9.533349 GB avg = 0.165 KB (by entry num) total key len = 1.597020 GB avg = 0.028 KB total ukey len = 1.109449 GB avg = 0.016 KB total ukey num = 0.071128972 Billion total entry num = 0.057936701 Billion write speed all = 0.277396249 MB/sec (with seq num) write speed all = 0.265844845 MB/sec (without seq num) 2024/11/23-20:03:56.701117 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278474: 928905 keys, 37077838 bytes, temperature: kWarm 2024/11/23-20:03:56.701189 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363436701152, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278474, "file_size": 37077838, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776210, "table_properties": {"data_size": 30846240, "index_size": 1778624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363433, "oldest_key_time": 0, "file_creation_time": 1732363433, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278474, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:03:56.737014 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278475.sst) /dev/shm/ToplingTemp/Topling-174138-3ZH49G 2024/11/23-20:03:57.741059 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278475.sst): first pass time = 1.00's, 133.681'MB/sec 2024/11/23-20:03:59.097412 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278475.sst): old prealloc_size = 147639500, real_size = 32410624 2024/11/23-20:03:59.098033 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278475.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934040 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.70 avg-zval = 32.53 2024/11/23-20:03:59.098847 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278475.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:03:59.962241 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278475.sst): second pass time = 1.36's, 79.719'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.376'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3006.243'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 280.527'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33024.226'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.719'MB/sec zip pipeline throughput = 79.719'MB/sec entries = 934040 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.70 avg-zval = 32.53 usrkeys = 934040 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.70 avg-zval = 32.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934040 bytes = 3385904 } seq expand size = 6538280 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4451 value = 3.5561 dict = 1.00 all = 3.6624 } Zip/UnZip{ index = 0.0957 value = 0.2812 dict = 1.00 all = 0.2730 } ---------------------------- total value len = 9.647952 GB avg = 0.164 KB (by entry num) total key len = 1.623173 GB avg = 0.028 KB total ukey len = 1.128130 GB avg = 0.016 KB total ukey num = 0.072063012 Billion total entry num = 0.058870741 Billion write speed all = 0.280881079 MB/sec (with seq num) write speed all = 0.269144414 MB/sec (without seq num) 2024/11/23-20:04:00.007884 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278475: 934040 keys, 36647186 bytes, temperature: kWarm 2024/11/23-20:04:00.007956 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363440007920, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278475, "file_size": 36647186, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30388464, "index_size": 1788480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26153120, "raw_average_key_size": 28, "raw_value_size": 108064620, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934040, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363436, "oldest_key_time": 0, "file_creation_time": 1732363436, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278475, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:00.046420 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278476.sst) /dev/shm/ToplingTemp/Topling-174138-2hMnRG 2024/11/23-20:04:01.014628 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278476.sst): first pass time = 0.97's, 138.632'MB/sec 2024/11/23-20:04:02.345393 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278476.sst): old prealloc_size = 147639500, real_size = 34668480 2024/11/23-20:04:02.346003 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278476.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931792 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0327 GB avg-val = 116.04 avg-zval = 35.04 2024/11/23-20:04:02.346825 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278476.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:04:03.187022 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278476.sst): second pass time = 1.33's, 81.319'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.356'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3082.278'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 302.840'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32340.136'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.319'MB/sec zip pipeline throughput = 81.319'MB/sec entries = 931792 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.04 avg-zval = 35.04 usrkeys = 931792 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.04 avg-zval = 35.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931792 bytes = 3494240 } seq expand size = 6522544 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0327 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4435 value = 3.3116 dict = 1.00 all = 3.4403 } Zip/UnZip{ index = 0.0958 value = 0.3020 dict = 1.00 all = 0.2907 } ---------------------------- total value len = 9.762602 GB avg = 0.163 KB (by entry num) total key len = 1.649263 GB avg = 0.028 KB total ukey len = 1.146765 GB avg = 0.016 KB total ukey num = 0.072994804 Billion total entry num = 0.059802533 Billion write speed all = 0.284365535 MB/sec (with seq num) write speed all = 0.272444063 MB/sec (without seq num) 2024/11/23-20:04:03.235702 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278476: 931792 keys, 39013270 bytes, temperature: kWarm 2024/11/23-20:04:03.235772 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363443235736, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278476, "file_size": 39013270, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 32650928, "index_size": 1784448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26090176, "raw_average_key_size": 28, "raw_value_size": 108127668, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931792, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363440, "oldest_key_time": 0, "file_creation_time": 1732363440, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278476, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:03.272404 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278477.sst) /dev/shm/ToplingTemp/Topling-174138-3IJpfF 2024/11/23-20:04:04.193902 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278477.sst): first pass time = 0.92's, 145.658'MB/sec 2024/11/23-20:04:05.477597 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278477.sst): old prealloc_size = 147639500, real_size = 30287040 2024/11/23-20:04:05.478195 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278477.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930366 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0283 GB avg-val = 116.26 avg-zval = 30.38 2024/11/23-20:04:05.479005 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278477.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:04:06.296604 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278477.sst): second pass time = 1.28's, 84.313'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.279'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3115.640'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 269.591'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34338.002'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.313'MB/sec zip pipeline throughput = 84.313'MB/sec entries = 930366 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.26 avg-zval = 30.38 usrkeys = 930366 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.26 avg-zval = 30.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930366 bytes = 3488880 } seq expand size = 6512562 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0283 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.3765 value = 3.8274 dict = 1.00 all = 3.8762 } Zip/UnZip{ index = 0.0964 value = 0.2613 dict = 1.00 all = 0.2580 } ---------------------------- total value len = 9.877282 GB avg = 0.163 KB (by entry num) total key len = 1.675314 GB avg = 0.028 KB total ukey len = 1.165373 GB avg = 0.016 KB total ukey num = 0.073925170 Billion total entry num = 0.060732899 Billion write speed all = 0.287850007 MB/sec (with seq num) write speed all = 0.275744007 MB/sec (without seq num) 2024/11/23-20:04:06.342304 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278477: 930366 keys, 34625974 bytes, temperature: kWarm 2024/11/23-20:04:06.342377 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363446342338, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278477, "file_size": 34625974, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3426788746, "largest_seqno": 4012776300, "table_properties": {"data_size": 28261040, "index_size": 1793216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26050248, "raw_average_key_size": 28, "raw_value_size": 108167560, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930366, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363443, "oldest_key_time": 0, "file_creation_time": 1732363443, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278477, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:06.385781 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278478.sst) /dev/shm/ToplingTemp/Topling-174138-vLOauJ 2024/11/23-20:04:07.353955 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278478.sst): first pass time = 0.97's, 138.635'MB/sec 2024/11/23-20:04:08.680692 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278478.sst): old prealloc_size = 147639500, real_size = 35258208 2024/11/23-20:04:08.681300 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278478.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923573 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0333 GB avg-val = 117.32 avg-zval = 36.02 2024/11/23-20:04:08.682099 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278478.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:04:09.520201 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278478.sst): second pass time = 1.33's, 81.720'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.147'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2985.226'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 311.341'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34596.032'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.720'MB/sec zip pipeline throughput = 81.720'MB/sec entries = 923573 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.32 avg-zval = 36.02 usrkeys = 923573 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.32 avg-zval = 36.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923573 bytes = 3463408 } seq expand size = 6465011 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5135 value = 3.2569 dict = 1.00 all = 3.3911 } Zip/UnZip{ index = 0.0951 value = 0.3070 dict = 1.00 all = 0.2949 } ---------------------------- total value len = 9.992105 GB avg = 0.162 KB (by entry num) total key len = 1.701174 GB avg = 0.028 KB total ukey len = 1.183844 GB avg = 0.016 KB total ukey num = 0.074848743 Billion total entry num = 0.061656472 Billion write speed all = 0.291331926 MB/sec (with seq num) write speed all = 0.279042816 MB/sec (without seq num) 2024/11/23-20:04:09.567324 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278478: 923573 keys, 39579280 bytes, temperature: kWarm 2024/11/23-20:04:09.567394 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363449567358, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278478, "file_size": 39579280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33270224, "index_size": 1756928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25860044, "raw_average_key_size": 28, "raw_value_size": 108357694, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923573, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363446, "oldest_key_time": 0, "file_creation_time": 1732363446, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278478, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:09.608267 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278479.sst) /dev/shm/ToplingTemp/Topling-174138-Q2Ra6E 2024/11/23-20:04:10.619868 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278479.sst): first pass time = 1.01's, 132.683'MB/sec 2024/11/23-20:04:11.928602 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278479.sst): old prealloc_size = 147639500, real_size = 31859168 2024/11/23-20:04:11.929173 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278479.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0298 GB avg-val = 116.15 avg-zval = 32.05 2024/11/23-20:04:11.929993 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278479.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:04:12.772281 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278479.sst): second pass time = 1.31's, 82.687'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.912'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3247.777'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 278.806'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31849.984'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.687'MB/sec zip pipeline throughput = 82.687'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.15 avg-zval = 32.05 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.15 avg-zval = 32.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4434 value = 3.6239 dict = 1.00 all = 3.7074 } Zip/UnZip{ index = 0.0958 value = 0.2759 dict = 1.00 all = 0.2697 } ---------------------------- total value len = 10.106770 GB avg = 0.161 KB (by entry num) total key len = 1.727244 GB avg = 0.028 KB total ukey len = 1.202466 GB avg = 0.016 KB total ukey num = 0.075779826 Billion total entry num = 0.062587555 Billion write speed all = 0.294814389 MB/sec (with seq num) write speed all = 0.282340710 MB/sec (without seq num) 2024/11/23-20:04:12.816660 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278479: 931083 keys, 36202520 bytes, temperature: kWarm 2024/11/23-20:04:12.816729 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363452816694, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278479, "file_size": 36202520, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 29843088, "index_size": 1783104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363449, "oldest_key_time": 0, "file_creation_time": 1732363449, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278479, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:12.854636 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278480.sst) /dev/shm/ToplingTemp/Topling-174138-ZpJecJ 2024/11/23-20:04:13.683334 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278480.sst): first pass time = 0.83's, 161.969'MB/sec 2024/11/23-20:04:14.994418 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278480.sst): old prealloc_size = 147639500, real_size = 31960576 2024/11/23-20:04:14.994945 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278480.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925740 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0300 GB avg-val = 116.98 avg-zval = 32.36 2024/11/23-20:04:14.995701 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278480.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:04:15.819758 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278480.sst): second pass time = 1.31's, 82.652'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.611'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3518.382'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 275.938'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30433.271'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.652'MB/sec zip pipeline throughput = 82.652'MB/sec entries = 925740 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.98 avg-zval = 32.36 usrkeys = 925740 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.30 avg-zval = 32.36 TagRS{ kinds = RS_Key0_TagN bytes = 144760 } TagArray{ size = 835356 bytes = 3028176 } seq expand size = 5847492 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4438 value = 3.6152 dict = 1.00 all = 3.7290 } Zip/UnZip{ index = 0.0958 value = 0.2766 dict = 1.00 all = 0.2682 } ---------------------------- total value len = 10.220915 GB avg = 0.161 KB (by entry num) total key len = 1.753165 GB avg = 0.028 KB total ukey len = 1.220981 GB avg = 0.016 KB total ukey num = 0.076705566 Billion total entry num = 0.063513295 Billion write speed all = 0.298281116 MB/sec (with seq num) write speed all = 0.285623898 MB/sec (without seq num) 2024/11/23-20:04:15.863845 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278480: 925740 keys, 35992906 bytes, temperature: kWarm 2024/11/23-20:04:15.863921 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363455863880, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278480, "file_size": 35992906, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 29956144, "index_size": 1772800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25920720, "raw_average_key_size": 28, "raw_value_size": 108297148, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925740, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363452, "oldest_key_time": 0, "file_creation_time": 1732363452, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278480, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:15.900280 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278481.sst) /dev/shm/ToplingTemp/Topling-174138-1m2sDJ 2024/11/23-20:04:16.882758 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278481.sst): first pass time = 0.98's, 136.617'MB/sec 2024/11/23-20:04:18.437169 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278481.sst): old prealloc_size = 147639500, real_size = 36863632 2024/11/23-20:04:18.437691 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278481.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926242 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 116.91 avg-zval = 37.63 2024/11/23-20:04:18.437869 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278481.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:04:19.146259 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278481.sst): second pass time = 1.55's, 69.694'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 53.800'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3506.523'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.396'MB/sec write SST data time = 0.12's, 309.384'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35002.592'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.694'MB/sec zip pipeline throughput = 69.694'MB/sec entries = 926242 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.91 avg-zval = 37.63 usrkeys = 926242 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.58 avg-zval = 37.63 TagRS{ kinds = RS_Key0_TagN bytes = 144840 } TagArray{ size = 89638 bytes = 280128 } seq expand size = 627466 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4450 value = 3.1064 dict = 1.00 all = 3.5184 } Zip/UnZip{ index = 0.0957 value = 0.3219 dict = 1.00 all = 0.2842 } ---------------------------- total value len = 10.329825 GB avg = 0.160 KB (by entry num) total key len = 1.779100 GB avg = 0.028 KB total ukey len = 1.239505 GB avg = 0.016 KB total ukey num = 0.077631808 Billion total entry num = 0.064439537 Billion write speed all = 0.301615194 MB/sec (with seq num) write speed all = 0.288774454 MB/sec (without seq num) 2024/11/23-20:04:19.192111 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278481: 926242 keys, 38146965 bytes, temperature: kWarm 2024/11/23-20:04:19.192181 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363459192145, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278481, "file_size": 38146965, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3267624809, "table_properties": {"data_size": 34858304, "index_size": 1773568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25934776, "raw_average_key_size": 28, "raw_value_size": 108283004, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926242, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363455, "oldest_key_time": 0, "file_creation_time": 1732363455, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278481, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:19.226903 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278482.sst) /dev/shm/ToplingTemp/Topling-174138-15nOwF 2024/11/23-20:04:20.228866 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278482.sst): first pass time = 1.00's, 133.960'MB/sec 2024/11/23-20:04:21.607640 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278482.sst): old prealloc_size = 147639500, real_size = 35779968 2024/11/23-20:04:21.608179 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278482.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923696 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0338 GB avg-val = 117.31 avg-zval = 36.57 2024/11/23-20:04:21.608738 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278482.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:04:22.385693 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278482.sst): second pass time = 1.38's, 78.632'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 49.927'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3425.203'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.354'MB/sec write SST data time = 0.13's, 310.271'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 32083.121'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.632'MB/sec zip pipeline throughput = 78.632'MB/sec entries = 923696 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.31 avg-zval = 36.57 usrkeys = 923696 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.47 avg-zval = 36.57 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 549091 bytes = 2059104 } seq expand size = 3843637 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.4456 value = 3.2076 dict = 1.00 all = 3.4553 } Zip/UnZip{ index = 0.0957 value = 0.3118 dict = 1.00 all = 0.2894 } ---------------------------- total value len = 10.442023 GB avg = 0.160 KB (by entry num) total key len = 1.804963 GB avg = 0.028 KB total ukey len = 1.257979 GB avg = 0.016 KB total ukey num = 0.078555504 Billion total entry num = 0.065363233 Billion write speed all = 0.305029486 MB/sec (with seq num) write speed all = 0.292005734 MB/sec (without seq num) 2024/11/23-20:04:22.435036 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278482: 923696 keys, 38844548 bytes, temperature: kWarm 2024/11/23-20:04:22.435108 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363462435071, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278482, "file_size": 38844548, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4232675464, "table_properties": {"data_size": 33780272, "index_size": 1768576, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25863488, "raw_average_key_size": 28, "raw_value_size": 108354352, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923696, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363459, "oldest_key_time": 0, "file_creation_time": 1732363459, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278482, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:22.471126 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278483.sst) /dev/shm/ToplingTemp/Topling-174138-besd0G 2024/11/23-20:04:23.517036 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278483.sst): first pass time = 1.05's, 128.331'MB/sec 2024/11/23-20:04:24.852966 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278483.sst): old prealloc_size = 147639500, real_size = 34047120 2024/11/23-20:04:24.853527 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278483.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931082 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0320 GB avg-val = 116.15 avg-zval = 34.41 2024/11/23-20:04:24.854262 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278483.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:04:25.695937 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278483.sst): second pass time = 1.34's, 81.003'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.128'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3428.476'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.12's, 308.832'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35234.616'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.003'MB/sec zip pipeline throughput = 81.003'MB/sec entries = 931082 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.15 avg-zval = 34.41 usrkeys = 931082 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 34.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931082 bytes = 3142416 } seq expand size = 6517574 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5132 value = 3.3751 dict = 1.00 all = 3.5283 } Zip/UnZip{ index = 0.0951 value = 0.2963 dict = 1.00 all = 0.2834 } ---------------------------- total value len = 10.556688 GB avg = 0.159 KB (by entry num) total key len = 1.831033 GB avg = 0.028 KB total ukey len = 1.276601 GB avg = 0.016 KB total ukey num = 0.079486586 Billion total entry num = 0.066294315 Billion write speed all = 0.308509274 MB/sec (with seq num) write speed all = 0.295301091 MB/sec (without seq num) 2024/11/23-20:04:25.743367 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278483: 931082 keys, 38040720 bytes, temperature: kWarm 2024/11/23-20:04:25.743437 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363465743402, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278483, "file_size": 38040720, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4232676549, "table_properties": {"data_size": 32042880, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070296, "raw_average_key_size": 28, "raw_value_size": 108147440, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931082, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363462, "oldest_key_time": 0, "file_creation_time": 1732363462, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278483, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:25.782917 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278484.sst) /dev/shm/ToplingTemp/Topling-174138-wECRBJ 2024/11/23-20:04:26.837652 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278484.sst): first pass time = 1.05's, 127.258'MB/sec 2024/11/23-20:04:28.152863 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278484.sst): old prealloc_size = 147639500, real_size = 35368560 2024/11/23-20:04:28.153446 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278484.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931067 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0334 GB avg-val = 116.15 avg-zval = 35.82 2024/11/23-20:04:28.154778 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278484.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:04:28.982103 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278484.sst): second pass time = 1.31's, 82.285'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.469'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3166.909'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 314.446'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31986.711'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.285'MB/sec zip pipeline throughput = 82.285'MB/sec entries = 931067 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.15 avg-zval = 35.82 usrkeys = 931067 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.15 avg-zval = 35.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931067 bytes = 3142368 } seq expand size = 6517469 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4458 value = 3.2425 dict = 1.00 all = 3.4098 } Zip/UnZip{ index = 0.0957 value = 0.3084 dict = 1.00 all = 0.2933 } ---------------------------- total value len = 10.671353 GB avg = 0.159 KB (by entry num) total key len = 1.857103 GB avg = 0.028 KB total ukey len = 1.295222 GB avg = 0.016 KB total ukey num = 0.080417653 Billion total entry num = 0.067225382 Billion write speed all = 0.311988673 MB/sec (with seq num) write speed all = 0.298596084 MB/sec (without seq num) 2024/11/23-20:04:29.028928 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278484: 931067 keys, 39362320 bytes, temperature: kWarm 2024/11/23-20:04:29.029009 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363469028962, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278484, "file_size": 39362320, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4103925250, "largest_seqno": 4232676001, "table_properties": {"data_size": 33352928, "index_size": 1782656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26069876, "raw_average_key_size": 28, "raw_value_size": 108147886, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931067, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363465, "oldest_key_time": 0, "file_creation_time": 1732363465, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278484, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:29.069580 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278485.sst) /dev/shm/ToplingTemp/Topling-174138-38xgpG 2024/11/23-20:04:30.112888 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278485.sst): first pass time = 1.04's, 128.651'MB/sec 2024/11/23-20:04:31.420380 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278485.sst): old prealloc_size = 147639500, real_size = 34009008 2024/11/23-20:04:31.420894 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278485.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930071 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0320 GB avg-val = 116.31 avg-zval = 34.41 2024/11/23-20:04:31.421648 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278485.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:04:32.262202 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278485.sst): second pass time = 1.31's, 82.786'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.146'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3605.803'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.12's, 304.817'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33843.624'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.786'MB/sec zip pipeline throughput = 82.786'MB/sec entries = 930071 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.31 avg-zval = 34.41 usrkeys = 930071 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.31 avg-zval = 34.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930071 bytes = 3139008 } seq expand size = 6510497 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5109 value = 3.3798 dict = 1.00 all = 3.5317 } Zip/UnZip{ index = 0.0951 value = 0.2959 dict = 1.00 all = 0.2831 } ---------------------------- total value len = 10.786040 GB avg = 0.158 KB (by entry num) total key len = 1.883145 GB avg = 0.028 KB total ukey len = 1.313824 GB avg = 0.016 KB total ukey num = 0.081347724 Billion total entry num = 0.068155453 Billion write speed all = 0.315467379 MB/sec (with seq num) write speed all = 0.301890612 MB/sec (without seq num) 2024/11/23-20:04:32.308541 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278485: 930071 keys, 38003672 bytes, temperature: kWarm 2024/11/23-20:04:32.308607 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363472308572, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278485, "file_size": 38003672, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4159804009, "largest_seqno": 4232676612, "table_properties": {"data_size": 32006560, "index_size": 1769728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26041988, "raw_average_key_size": 28, "raw_value_size": 108175874, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930071, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363469, "oldest_key_time": 0, "file_creation_time": 1732363469, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278485, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:32.343970 139957584442944 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278486.sst) /dev/shm/ToplingTemp/Topling-174138-qenycI 2024/11/23-20:04:32.473570 139957584442944 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278486.sst): first pass time = 0.13's, 244.047'MB/sec 2024/11/23-20:04:32.773538 139957584442944 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278486.sst): old prealloc_size = 147639500, real_size = 8897680 2024/11/23-20:04:32.773720 139957584442944 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278486.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 218184 prefix = 4 raw-key = 0.0035 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0255 GB zip-val = 0.0084 GB avg-val = 116.91 avg-zval = 38.62 2024/11/23-20:04:32.773931 139957584442944 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278486.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.032 GB 2024/11/23-20:04:32.968986 139957584442944 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00/278486.sst): second pass time = 0.30's, 85.220'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 50.384'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2641.520'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.332'MB/sec write SST data time = 0.03's, 318.136'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.264'MB, throughput = 8565.553'MB/sec zip my value time = 0.30's, unzip length = 0.026'GB zip my value throughput = 85.220'MB/sec zip pipeline throughput = 85.220'MB/sec entries = 218184 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.91 avg-zval = 38.62 usrkeys = 218184 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.91 avg-zval = 38.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 218184 bytes = 736384 } seq expand size = 1527288 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0044 GB value = 0.0255 GB dict = 0.26 MB all = 0.0316 GB } __ZipSize{ index = 0.0004 GB value = 0.0084 GB dict = 0.26 MB all = 0.0098 GB } UnZip/Zip{ index = 10.4832 value = 3.0270 dict = 1.00 all = 3.2112 } Zip/UnZip{ index = 0.0954 value = 0.3304 dict = 1.00 all = 0.3114 } ---------------------------- total value len = 10.813075 GB avg = 0.158 KB (by entry num) total key len = 1.889254 GB avg = 0.028 KB total ukey len = 1.318187 GB avg = 0.016 KB total ukey num = 0.081565908 Billion total entry num = 0.068373637 Billion write speed all = 0.316287114 MB/sec (with seq num) write speed all = 0.302667123 MB/sec (without seq num) 2024/11/23-20:04:32.986397 139957584442944 [compaction_job.cc:1948] [default] [JOB 34781] Generated table #278486: 218184 keys, 9845915 bytes, temperature: kWarm 2024/11/23-20:04:32.986457 139957584442944 EVENT_LOG_v1 {"time_micros": 1732363472986425, "cf_name": "default", "job": 34781, "event": "table_file_creation", "file_number": 278486, "file_size": 9845915, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4160310309, "largest_seqno": 4232676573, "table_properties": {"data_size": 8426688, "index_size": 416256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6109152, "raw_average_key_size": 28, "raw_value_size": 25507692, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 218184, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732363472, "oldest_key_time": 0, "file_creation_time": 1732363472, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278486, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:04:32.999229 139957584442944 [compaction_job.cc:703] job-34781: subcompact[0], size: 0.659905 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34781/att-00 [278469,278470,278471,278472,278473,278474,278475,278476,278477,278478,278479,278480,278481,278482,278483,278484,278485,278486] 2024/11/23-20:04:33.110016 139957584442944 (Original Log Time 2024/11/23-20:03:35.421005) EVENT_LOG_v1 {"time_micros": 1732363415420959, "job": 34781, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [278408], "files_L3": [278435, 278436, 278363, 278364, 278365, 278366, 278367, 278368, 278369, 278370, 278371, 278372, 278373, 278374, 278375, 278376, 278377, 278437], "score": 0.989473, "input_data_size": 683077275, "oldest_snapshot_seqno": 4233070534} 2024/11/23-20:04:33.110020 139957584442944 (Original Log Time 2024/11/23-20:04:33.108524) [compaction_job.cc:2022] [default] [JOB 34781] Compacted 1@2 + 18@3 files to L3 => 659904648 bytes 2024/11/23-20:04:33.110021 139957584442944 (Original Log Time 2024/11/23-20:04:33.109944) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 18 0 0 0] max score 0.02, MB/sec: 11.9 rd, 11.5 wr, level 3, files in(1, 18) out(18 +0 blob) MB in(27.1, 624.3 +0.0 blob) out(629.3 +0.0 blob), read-write-amplify(47.3) write-amplify(23.2) OK, records in: 16090158, records dropped: 74685 output_compression: Snappy 2024/11/23-20:04:33.110026 139957584442944 (Original Log Time 2024/11/23-20:04:33.110004) EVENT_LOG_v1 {"time_micros": 1732363473109966, "cf": "default", "job": 34781, "event": "compaction_finished", "compaction_time_micros": 57578231, "compaction_time_cpu_micros": 23994078, "output_level": 3, "num_output_files": 18, "total_output_size": 659904648, "num_input_records": 16090158, "num_output_records": 16015473, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 454076646, "file_range_sync_nanos": 0, "file_fsync_nanos": 12386, "file_prepare_write_nanos": 54540, "lsm_state": [0, 0, 0, 18, 0, 0, 0]} 2024/11/23-20:04:33.253670 139957584442944 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732363415,"executesMs":57847,"compactionJobId":34781,"attempt":0,"compactionInputRawBytes":2323924832,"compactionInputZipBytes":683077275,"compactionOutputRawBytes":0,"compactionOutputZipBytes":659904648,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-5ceb5cd42106edb5643d9c1b55f8744e-ab0102fc89a1c4fd-00"}}, response = 2024/11/23-20:04:33.253729 139957584442944 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34781/att-00: olev 3, work 57.848 s, result 7.811 ms, install 1.519 ms, input{raw 2.164 GiB zip 651.433 MiB}