2024/11/23-15:13:30.731140 139957517301312 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00: fork to child time = 0.010409 sec 2024/11/23-15:13:30.731932 139957517301312 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00 2024/11/23-15:13:30.739778 139957517301312 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00 2024/11/23-15:13:30.740739 139957517301312 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-15:13:30.741011 139957517301312 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-15:13:30.796069 139957517301312 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28941/att-00/MANIFEST-206347 2024/11/23-15:13:30.833513 139957517301312 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28941/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 265783, last_sequence is 4057761006, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 265621 2024/11/23-15:13:30.833522 139957517301312 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-15:13:30.833667 139957517301312 [version_set.cc:5721] Creating manifest 265783 2024/11/23-15:13:30.869272 139957517301312 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-15:13:30.891301 139957517301312 [compaction_job.cc:2437] [default] [JOB 28941] Compacting 1@2 + 23@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-15:13:30.891320 139957517301312 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 21635 Base level 2, inputs: [265775(21MB)], [265729(39MB) 265730(38MB) 265731(35MB) 265732(34MB) 265733(34MB) 265734(33MB) 265735(34MB) 265736(36MB) 265737(35MB) 265738(35MB) 265739(35MB) 265740(36MB) 265741(37MB) 265742(37MB) 265743(35MB) 265744(34MB) 265745(37MB) 265746(33MB) 265747(37MB) 265748(34MB) 265749(34MB) 265750(36MB) 265751(37MB)] 2024/11/23-15:13:30.897367 139957517301312 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:13:30.897392 139957517301312 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:13:30.897407 139957517301312 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-15:13:30.897412 139957517301312 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:13:30.897423 139957517301312 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-15:13:31.438836 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265784.sst) /dev/shm/ToplingTemp/Topling-89534-vfSU9j 2024/11/23-15:13:32.288415 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265784.sst): first pass time = 0.85's, 157.992'MB/sec 2024/11/23-15:13:35.819730 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265784.sst): old prealloc_size = 147639500, real_size = 37334816 2024/11/23-15:13:35.820414 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265784.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928074 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0353 GB avg-val = 116.62 avg-zval = 38.08 2024/11/23-15:13:35.821248 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265784.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:13:37.279085 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265784.sst): second pass time = 3.53's, 30.665'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.46's, 28.235'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2659.379'MB/sec (index lex order gen) rebuild zvType time = 1.16's, 0.201'MB/sec write SST data time = 0.30's, 136.402'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33184.825'MB/sec zip my value time = 3.53's, unzip length = 0.108'GB zip my value throughput = 30.665'MB/sec zip pipeline throughput = 78.971'MB/sec entries = 928074 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.62 avg-zval = 38.08 usrkeys = 928074 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.62 avg-zval = 38.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928074 bytes = 3016256 } seq expand size = 6496518 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0353 GB dict = 1.08 MB all = 0.0412 GB } UnZip/Zip{ index = 10.5149 value = 3.0628 dict = 1.00 all = 3.2574 } Zip/UnZip{ index = 0.0951 value = 0.3265 dict = 1.00 all = 0.3070 } ---------------------------- total value len = 8.963348 GB avg = 0.168 KB (by entry num) total key len = 1.466807 GB avg = 0.028 KB total ukey len = 1.016439 GB avg = 0.015 KB total ukey num = 0.066478509 Billion total entry num = 0.053286238 Billion write speed all = 0.459375712 MB/sec (with seq num) write speed all = 0.440600610 MB/sec (without seq num) 2024/11/23-15:13:37.385224 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265784: 928074 keys, 41204316 bytes, temperature: kWarm 2024/11/23-15:13:37.385315 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346017385260, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265784, "file_size": 41204316, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4012777911, "largest_seqno": 4057728128, "table_properties": {"data_size": 35337360, "index_size": 1765248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986072, "raw_average_key_size": 28, "raw_value_size": 108231664, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928074, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346011, "oldest_key_time": 0, "file_creation_time": 1732346011, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265784, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:13:37.431413 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265785.sst) /dev/shm/ToplingTemp/Topling-89534-3B9p7i 2024/11/23-15:13:38.369506 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265785.sst): first pass time = 0.94's, 143.082'MB/sec 2024/11/23-15:13:41.223110 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265785.sst): old prealloc_size = 147639500, real_size = 36662016 2024/11/23-15:13:41.224223 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265785.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928180 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0347 GB avg-val = 116.60 avg-zval = 37.33 2024/11/23-15:13:41.225521 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265785.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:13:42.420604 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265785.sst): second pass time = 2.85's, 37.938'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.20's, 34.142'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1641.659'MB/sec (index lex order gen) rebuild zvType time = 1.04's, 0.223'MB/sec write SST data time = 0.16's, 259.589'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35671.368'MB/sec zip my value time = 2.85's, unzip length = 0.108'GB zip my value throughput = 37.938'MB/sec zip pipeline throughput = 62.627'MB/sec entries = 928180 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.60 avg-zval = 37.33 usrkeys = 928180 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.60 avg-zval = 37.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928180 bytes = 3364672 } seq expand size = 6497260 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0409 GB } UnZip/Zip{ index = 10.4446 value = 3.1233 dict = 1.00 all = 3.2829 } Zip/UnZip{ index = 0.0957 value = 0.3202 dict = 1.00 all = 0.3046 } ---------------------------- total value len = 9.078074 GB avg = 0.167 KB (by entry num) total key len = 1.492796 GB avg = 0.028 KB total ukey len = 1.035003 GB avg = 0.015 KB total ukey num = 0.067406689 Billion total entry num = 0.054214418 Billion write speed all = 0.465467824 MB/sec (with seq num) write speed all = 0.446370007 MB/sec (without seq num) 2024/11/23-15:13:42.529486 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265785: 928180 keys, 40884262 bytes, temperature: kWarm 2024/11/23-15:13:42.529547 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346022529518, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265785, "file_size": 40884262, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 4057725933, "table_properties": {"data_size": 34652464, "index_size": 1777344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989040, "raw_average_key_size": 28, "raw_value_size": 108228716, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928180, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346017, "oldest_key_time": 0, "file_creation_time": 1732346017, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265785, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:13:42.572652 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265786.sst) /dev/shm/ToplingTemp/Topling-89534-TtI0Ik 2024/11/23-15:13:43.762294 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265786.sst): first pass time = 1.19's, 112.825'MB/sec 2024/11/23-15:13:47.084243 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265786.sst): old prealloc_size = 147639500, real_size = 32073504 2024/11/23-15:13:47.084792 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265786.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931352 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0301 GB avg-val = 116.11 avg-zval = 32.27 2024/11/23-15:13:47.085663 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265786.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:13:48.083275 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265786.sst): second pass time = 3.32's, 32.560'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.00's, 36.455'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3375.072'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.23's, 155.572'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31577.721'MB/sec zip my value time = 3.32's, unzip length = 0.108'GB zip my value throughput = 32.560'MB/sec zip pipeline throughput = 110.385'MB/sec entries = 931352 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.11 avg-zval = 32.27 usrkeys = 931352 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.11 avg-zval = 32.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931352 bytes = 3492592 } seq expand size = 6519464 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4475 value = 3.5978 dict = 1.00 all = 3.6855 } Zip/UnZip{ index = 0.0957 value = 0.2780 dict = 1.00 all = 0.2713 } ---------------------------- total value len = 9.192733 GB avg = 0.167 KB (by entry num) total key len = 1.518874 GB avg = 0.028 KB total ukey len = 1.053630 GB avg = 0.015 KB total ukey num = 0.068338041 Billion total entry num = 0.055145770 Billion write speed all = 0.471547341 MB/sec (with seq num) write speed all = 0.452126284 MB/sec (without seq num) 2024/11/23-15:13:48.130925 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265786: 931352 keys, 36417468 bytes, temperature: kWarm 2024/11/23-15:13:48.131025 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346028130966, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265786, "file_size": 36417468, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 4012776133, "table_properties": {"data_size": 30057552, "index_size": 1782912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26077856, "raw_average_key_size": 28, "raw_value_size": 108139968, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931352, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346022, "oldest_key_time": 0, "file_creation_time": 1732346022, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265786, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:13:48.202423 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265787.sst) /dev/shm/ToplingTemp/Topling-89534-o8Xoal 2024/11/23-15:13:49.316576 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265787.sst): first pass time = 1.11's, 120.471'MB/sec 2024/11/23-15:13:50.843777 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265787.sst): old prealloc_size = 147639500, real_size = 33079152 2024/11/23-15:13:50.844423 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265787.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930477 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.25 avg-zval = 33.40 2024/11/23-15:13:50.845225 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265787.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:13:51.707496 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265787.sst): second pass time = 1.53's, 70.861'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.191'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2835.747'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.13's, 295.140'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 36229.165'MB/sec zip my value time = 1.53's, unzip length = 0.108'GB zip my value throughput = 70.861'MB/sec zip pipeline throughput = 70.861'MB/sec entries = 930477 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.25 avg-zval = 33.40 usrkeys = 930477 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.25 avg-zval = 33.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930477 bytes = 3372992 } seq expand size = 6513339 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5136 value = 3.4806 dict = 1.00 all = 3.5980 } Zip/UnZip{ index = 0.0951 value = 0.2873 dict = 1.00 all = 0.2779 } ---------------------------- total value len = 9.307411 GB avg = 0.166 KB (by entry num) total key len = 1.544927 GB avg = 0.028 KB total ukey len = 1.072240 GB avg = 0.015 KB total ukey num = 0.069268518 Billion total entry num = 0.056076247 Billion write speed all = 0.477666411 MB/sec (with seq num) write speed all = 0.457920813 MB/sec (without seq num) 2024/11/23-15:13:51.753530 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265787: 930477 keys, 37303930 bytes, temperature: kWarm 2024/11/23-15:13:51.753599 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346031753564, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265787, "file_size": 37303930, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 4012776165, "table_properties": {"data_size": 31076320, "index_size": 1770048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26053356, "raw_average_key_size": 28, "raw_value_size": 108164486, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930477, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346028, "oldest_key_time": 0, "file_creation_time": 1732346028, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265787, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:13:51.790384 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265788.sst) /dev/shm/ToplingTemp/Topling-89534-RGrkVm 2024/11/23-15:13:52.962471 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265788.sst): first pass time = 1.17's, 114.516'MB/sec 2024/11/23-15:13:54.487508 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265788.sst): old prealloc_size = 147639500, real_size = 32387008 2024/11/23-15:13:54.488251 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265788.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934763 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0304 GB avg-val = 115.58 avg-zval = 32.49 2024/11/23-15:13:54.489137 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265788.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:13:55.427089 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265788.sst): second pass time = 1.50's, 71.850'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 38.857'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2470.231'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.300'MB/sec write SST data time = 0.16's, 229.357'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 38876.195'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 71.850'MB/sec zip pipeline throughput = 71.850'MB/sec entries = 934763 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.58 avg-zval = 32.49 usrkeys = 934763 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.58 avg-zval = 32.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934763 bytes = 3271680 } seq expand size = 6543341 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5141 value = 3.5570 dict = 1.00 all = 3.6763 } Zip/UnZip{ index = 0.0951 value = 0.2811 dict = 1.00 all = 0.2720 } ---------------------------- total value len = 9.421999 GB avg = 0.165 KB (by entry num) total key len = 1.571101 GB avg = 0.028 KB total ukey len = 1.090935 GB avg = 0.016 KB total ukey num = 0.070203281 Billion total entry num = 0.057011010 Billion write speed all = 0.483782818 MB/sec (with seq num) write speed all = 0.463711356 MB/sec (without seq num) 2024/11/23-15:13:55.471251 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265788: 934763 keys, 36508926 bytes, temperature: kWarm 2024/11/23-15:13:55.471312 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346035471280, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265788, "file_size": 36508926, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3759280178, "largest_seqno": 4012776176, "table_properties": {"data_size": 30375024, "index_size": 1778112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26173364, "raw_average_key_size": 28, "raw_value_size": 108044426, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934763, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346031, "oldest_key_time": 0, "file_creation_time": 1732346031, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265788, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:13:55.510636 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265789.sst) /dev/shm/ToplingTemp/Topling-89534-Gbw1Gm 2024/11/23-15:13:56.687690 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265789.sst): first pass time = 1.18's, 114.033'MB/sec 2024/11/23-15:13:58.171851 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265789.sst): old prealloc_size = 147639500, real_size = 32198832 2024/11/23-15:13:58.172438 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265789.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931885 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0302 GB avg-val = 116.03 avg-zval = 32.39 2024/11/23-15:13:58.173224 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265789.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:13:59.051491 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265789.sst): second pass time = 1.48's, 72.903'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.281'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3128.668'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.309'MB/sec write SST data time = 0.13's, 289.640'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36913.245'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 72.903'MB/sec zip pipeline throughput = 72.903'MB/sec entries = 931885 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.03 avg-zval = 32.39 usrkeys = 931885 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.03 avg-zval = 32.39 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931885 bytes = 3261616 } seq expand size = 6523195 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4453 value = 3.5825 dict = 1.00 all = 3.6963 } Zip/UnZip{ index = 0.0957 value = 0.2791 dict = 1.00 all = 0.2705 } ---------------------------- total value len = 9.536647 GB avg = 0.165 KB (by entry num) total key len = 1.597194 GB avg = 0.028 KB total ukey len = 1.109573 GB avg = 0.016 KB total ukey num = 0.071135166 Billion total entry num = 0.057942895 Billion write speed all = 0.489898391 MB/sec (with seq num) write speed all = 0.469502100 MB/sec (without seq num) 2024/11/23-15:13:59.104863 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265789: 931885 keys, 36311052 bytes, temperature: kWarm 2024/11/23-15:13:59.104930 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346039104896, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265789, "file_size": 36311052, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3801926538, "largest_seqno": 4012776192, "table_properties": {"data_size": 30181344, "index_size": 1784320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26092780, "raw_average_key_size": 28, "raw_value_size": 108125074, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931885, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346035, "oldest_key_time": 0, "file_creation_time": 1732346035, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265789, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:13:59.141260 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265790.sst) /dev/shm/ToplingTemp/Topling-89534-i5WGuk 2024/11/23-15:14:00.264563 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265790.sst): first pass time = 1.12's, 119.489'MB/sec 2024/11/23-15:14:02.023029 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265790.sst): old prealloc_size = 147639500, real_size = 34685216 2024/11/23-15:14:02.023711 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265790.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1085405 prefix = 4 raw-key = 0.0174 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1038 GB zip-val = 0.0323 GB avg-val = 95.66 avg-zval = 29.79 2024/11/23-15:14:02.024020 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265790.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:02.748074 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265790.sst): second pass time = 1.76's, 59.067'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.73's, 50.024'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3159.273'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.463'MB/sec write SST data time = 0.14's, 261.503'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.033'MB, throughput = 35667.599'MB/sec zip my value time = 1.76's, unzip length = 0.104'GB zip my value throughput = 59.067'MB/sec zip pipeline throughput = 117.062'MB/sec entries = 1085405 avg-key = 28.00 avg-zkey = 1.91 avg-val = 95.66 avg-zval = 29.79 usrkeys = 1085405 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.93 avg-zval = 29.79 TagRS{ kinds = RS_Key0_TagN bytes = 169680 } TagArray{ size = 197375 bytes = 641488 } seq expand size = 1381625 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0217 GB value = 0.1038 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0323 GB dict = 1.03 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4469 value = 3.2109 dict = 1.00 all = 3.7006 } Zip/UnZip{ index = 0.0957 value = 0.3114 dict = 1.00 all = 0.2702 } ---------------------------- total value len = 9.641855 GB avg = 0.163 KB (by entry num) total key len = 1.627585 GB avg = 0.028 KB total ukey len = 1.131281 GB avg = 0.016 KB total ukey num = 0.072220571 Billion total entry num = 0.059028300 Billion write speed all = 0.495784240 MB/sec (with seq num) write speed all = 0.475009259 MB/sec (without seq num) 2024/11/23-15:14:02.795686 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265790: 1085405 keys, 36269266 bytes, temperature: kWarm 2024/11/23-15:14:02.795762 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346042795722, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265790, "file_size": 36269266, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775259, "table_properties": {"data_size": 32335760, "index_size": 2077952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30391340, "raw_average_key_size": 28, "raw_value_size": 103826483, "raw_average_value_size": 95, "num_data_blocks": 1, "num_entries": 1085405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346039, "oldest_key_time": 0, "file_creation_time": 1732346039, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265790, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:02.833929 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265791.sst) /dev/shm/ToplingTemp/Topling-89534-bEOhPk 2024/11/23-15:14:03.847439 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265791.sst): first pass time = 1.01's, 132.435'MB/sec 2024/11/23-15:14:05.742629 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265791.sst): old prealloc_size = 147639500, real_size = 35908256 2024/11/23-15:14:05.743212 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265791.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923424 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0339 GB avg-val = 117.35 avg-zval = 36.73 2024/11/23-15:14:05.743664 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265791.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:06.656683 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265791.sst): second pass time = 1.89's, 57.198'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.122'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3134.901'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.339'MB/sec write SST data time = 0.23's, 165.104'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35077.716'MB/sec zip my value time = 1.89's, unzip length = 0.108'GB zip my value throughput = 57.198'MB/sec zip pipeline throughput = 126.676'MB/sec entries = 923424 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.35 avg-zval = 36.73 usrkeys = 923424 avg-key = 20.00 avg-zkey = 1.90 avg-val = 120.55 avg-zval = 36.73 TagRS{ kinds = RS_Key0_TagN bytes = 144400 } TagArray{ size = 423051 bytes = 1586464 } seq expand size = 2961357 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5130 value = 3.1946 dict = 1.00 all = 3.4861 } Zip/UnZip{ index = 0.0951 value = 0.3130 dict = 1.00 all = 0.2869 } ---------------------------- total value len = 9.753179 GB avg = 0.163 KB (by entry num) total key len = 1.653441 GB avg = 0.028 KB total ukey len = 1.149749 GB avg = 0.016 KB total ukey num = 0.073143995 Billion total entry num = 0.059951724 Billion write speed all = 0.501732984 MB/sec (with seq num) write speed all = 0.480636632 MB/sec (without seq num) 2024/11/23-15:14:06.710871 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265791: 923424 keys, 38500621 bytes, temperature: kWarm 2024/11/23-15:14:06.710963 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346046710915, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265791, "file_size": 38500621, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3968371688, "table_properties": {"data_size": 33920464, "index_size": 1756736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25855872, "raw_average_key_size": 28, "raw_value_size": 108361956, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346042, "oldest_key_time": 0, "file_creation_time": 1732346042, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265791, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:06.903649 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265792.sst) /dev/shm/ToplingTemp/Topling-89534-F4UG5j 2024/11/23-15:14:07.850312 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265792.sst): first pass time = 0.95's, 141.789'MB/sec 2024/11/23-15:14:09.360510 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265792.sst): old prealloc_size = 147639500, real_size = 32443952 2024/11/23-15:14:09.361106 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265792.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931429 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.10 avg-zval = 32.68 2024/11/23-15:14:09.362602 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265792.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:10.429833 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265792.sst): second pass time = 1.51's, 71.638'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 34.294'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3091.935'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.261'MB/sec write SST data time = 0.18's, 206.109'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35787.035'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.638'MB/sec zip pipeline throughput = 71.638'MB/sec entries = 931429 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.10 avg-zval = 32.68 usrkeys = 931429 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.10 avg-zval = 32.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931429 bytes = 3376448 } seq expand size = 6520003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5148 value = 3.5526 dict = 1.00 all = 3.6600 } Zip/UnZip{ index = 0.0951 value = 0.2815 dict = 1.00 all = 0.2732 } ---------------------------- total value len = 9.867836 GB avg = 0.162 KB (by entry num) total key len = 1.679521 GB avg = 0.028 KB total ukey len = 1.168378 GB avg = 0.016 KB total ukey num = 0.074075424 Billion total entry num = 0.060883153 Billion write speed all = 0.507839208 MB/sec (with seq num) write speed all = 0.486418652 MB/sec (without seq num) 2024/11/23-15:14:10.474919 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265792: 931429 keys, 36671580 bytes, temperature: kWarm 2024/11/23-15:14:10.475007 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346050474955, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265792, "file_size": 36671580, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 4012775885, "table_properties": {"data_size": 30439264, "index_size": 1771648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080012, "raw_average_key_size": 28, "raw_value_size": 108137762, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346046, "oldest_key_time": 0, "file_creation_time": 1732346046, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265792, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:10.512853 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265793.sst) /dev/shm/ToplingTemp/Topling-89534-jDRcqk 2024/11/23-15:14:11.479082 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265793.sst): first pass time = 0.97's, 138.915'MB/sec 2024/11/23-15:14:12.799036 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265793.sst): old prealloc_size = 147639500, real_size = 32392208 2024/11/23-15:14:12.799574 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265793.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931309 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 116.12 avg-zval = 32.62 2024/11/23-15:14:12.801042 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265793.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:13.645884 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265793.sst): second pass time = 1.32's, 81.983'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.106'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3449.971'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 290.462'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37156.680'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.983'MB/sec zip pipeline throughput = 81.983'MB/sec entries = 931309 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.12 avg-zval = 32.62 usrkeys = 931309 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.12 avg-zval = 32.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931309 bytes = 3259600 } seq expand size = 6519163 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4433 value = 3.5601 dict = 1.00 all = 3.6769 } Zip/UnZip{ index = 0.0958 value = 0.2809 dict = 1.00 all = 0.2720 } ---------------------------- total value len = 9.982497 GB avg = 0.161 KB (by entry num) total key len = 1.705597 GB avg = 0.028 KB total ukey len = 1.187004 GB avg = 0.016 KB total ukey num = 0.075006733 Billion total entry num = 0.061814462 Billion write speed all = 0.513955962 MB/sec (with seq num) write speed all = 0.492210818 MB/sec (without seq num) 2024/11/23-15:14:13.690241 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265793: 931309 keys, 36503108 bytes, temperature: kWarm 2024/11/23-15:14:13.690312 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346053690276, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265793, "file_size": 36503108, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775902, "table_properties": {"data_size": 30375616, "index_size": 1783552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26076652, "raw_average_key_size": 28, "raw_value_size": 108141150, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931309, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346050, "oldest_key_time": 0, "file_creation_time": 1732346050, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265793, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:13.727224 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265794.sst) /dev/shm/ToplingTemp/Topling-89534-ro7Tgm 2024/11/23-15:14:14.754442 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265794.sst): first pass time = 1.03's, 130.667'MB/sec 2024/11/23-15:14:16.061888 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265794.sst): old prealloc_size = 147639500, real_size = 32532400 2024/11/23-15:14:16.062468 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265794.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931462 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 116.09 avg-zval = 32.77 2024/11/23-15:14:16.064016 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265794.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:16.908491 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265794.sst): second pass time = 1.31's, 82.759'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.559'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3178.858'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 290.332'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32469.476'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.759'MB/sec zip pipeline throughput = 82.759'MB/sec entries = 931462 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.09 avg-zval = 32.77 usrkeys = 931462 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.09 avg-zval = 32.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931462 bytes = 3492992 } seq expand size = 6520234 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5129 value = 3.5423 dict = 1.00 all = 3.6397 } Zip/UnZip{ index = 0.0951 value = 0.2823 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 10.097154 GB avg = 0.161 KB (by entry num) total key len = 1.731678 GB avg = 0.028 KB total ukey len = 1.205633 GB avg = 0.016 KB total ukey num = 0.075938195 Billion total entry num = 0.062745924 Billion write speed all = 0.520069968 MB/sec (with seq num) write speed all = 0.498000320 MB/sec (without seq num) 2024/11/23-15:14:16.953671 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265794: 931462 keys, 36876500 bytes, temperature: kWarm 2024/11/23-15:14:16.953743 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346056953707, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265794, "file_size": 36876500, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775918, "table_properties": {"data_size": 30527328, "index_size": 1772032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26080936, "raw_average_key_size": 28, "raw_value_size": 108136860, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931462, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346053, "oldest_key_time": 0, "file_creation_time": 1732346053, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265794, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:16.992869 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265795.sst) /dev/shm/ToplingTemp/Topling-89534-l9LmDm 2024/11/23-15:14:18.043239 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265795.sst): first pass time = 1.05's, 127.786'MB/sec 2024/11/23-15:14:19.413366 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265795.sst): old prealloc_size = 147639500, real_size = 34012448 2024/11/23-15:14:19.413884 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265795.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925405 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0320 GB avg-val = 117.04 avg-zval = 34.60 2024/11/23-15:14:19.415970 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265795.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:20.256293 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265795.sst): second pass time = 1.37's, 79.092'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.346'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3541.935'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 298.805'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35279.713'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.092'MB/sec zip pipeline throughput = 79.092'MB/sec entries = 925405 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.04 avg-zval = 34.60 usrkeys = 925405 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.04 avg-zval = 34.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925405 bytes = 3354608 } seq expand size = 6477835 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0320 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5133 value = 3.3824 dict = 1.00 all = 3.5115 } Zip/UnZip{ index = 0.0951 value = 0.2956 dict = 1.00 all = 0.2848 } ---------------------------- total value len = 10.211938 GB avg = 0.160 KB (by entry num) total key len = 1.757590 GB avg = 0.028 KB total ukey len = 1.224141 GB avg = 0.016 KB total ukey num = 0.076863600 Billion total entry num = 0.063671329 Billion write speed all = 0.526178384 MB/sec (with seq num) write speed all = 0.503786538 MB/sec (without seq num) 2024/11/23-15:14:20.302574 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265795: 925405 keys, 38222314 bytes, temperature: kWarm 2024/11/23-15:14:20.302651 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346060302615, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265795, "file_size": 38222314, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3623676327, "largest_seqno": 4012775947, "table_properties": {"data_size": 32020496, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911340, "raw_average_key_size": 28, "raw_value_size": 108306414, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925405, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346056, "oldest_key_time": 0, "file_creation_time": 1732346056, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265795, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:20.339770 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265796.sst) /dev/shm/ToplingTemp/Topling-89534-emQHwm 2024/11/23-15:14:21.188773 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265796.sst): first pass time = 0.85's, 158.096'MB/sec 2024/11/23-15:14:22.526075 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265796.sst): old prealloc_size = 147639500, real_size = 35839040 2024/11/23-15:14:22.526643 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265796.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924151 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0338 GB avg-val = 117.23 avg-zval = 36.62 2024/11/23-15:14:22.528712 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265796.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:23.361272 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265796.sst): second pass time = 1.34's, 81.061'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.949'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3229.764'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 315.237'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33882.917'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.061'MB/sec zip pipeline throughput = 81.061'MB/sec entries = 924151 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.23 avg-zval = 36.62 usrkeys = 924151 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.23 avg-zval = 36.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924151 bytes = 3350064 } seq expand size = 6469057 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4451 value = 3.2017 dict = 1.00 all = 3.3516 } Zip/UnZip{ index = 0.0957 value = 0.3123 dict = 1.00 all = 0.2984 } ---------------------------- total value len = 10.326749 GB avg = 0.160 KB (by entry num) total key len = 1.783466 GB avg = 0.028 KB total ukey len = 1.242624 GB avg = 0.016 KB total ukey num = 0.077787751 Billion total entry num = 0.064595480 Billion write speed all = 0.532290298 MB/sec (with seq num) write speed all = 0.509576549 MB/sec (without seq num) 2024/11/23-15:14:23.410878 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265796: 924151 keys, 40045602 bytes, temperature: kWarm 2024/11/23-15:14:23.410947 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346063410909, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265796, "file_size": 40045602, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33838256, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25876228, "raw_average_key_size": 28, "raw_value_size": 108341550, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924151, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346060, "oldest_key_time": 0, "file_creation_time": 1732346060, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265796, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:23.448264 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265797.sst) /dev/shm/ToplingTemp/Topling-89534-pmHlrj 2024/11/23-15:14:24.303566 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265797.sst): first pass time = 0.86's, 156.932'MB/sec 2024/11/23-15:14:25.638299 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265797.sst): old prealloc_size = 147639500, real_size = 35187808 2024/11/23-15:14:25.638841 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265797.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930745 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0332 GB avg-val = 116.20 avg-zval = 35.64 2024/11/23-15:14:25.639571 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265797.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:26.450242 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265797.sst): second pass time = 1.33's, 81.082'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 48.025'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3445.745'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.340'MB/sec write SST data time = 0.13's, 307.898'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37182.094'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.082'MB/sec zip pipeline throughput = 81.082'MB/sec entries = 930745 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.20 avg-zval = 35.64 usrkeys = 930745 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.83 avg-zval = 35.64 TagRS{ kinds = RS_Key0_TagN bytes = 145520 } TagArray{ size = 747748 bytes = 2804064 } seq expand size = 5234236 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.4467 value = 3.2604 dict = 1.00 all = 3.4421 } Zip/UnZip{ index = 0.0957 value = 0.3067 dict = 1.00 all = 0.2905 } ---------------------------- total value len = 10.440140 GB avg = 0.159 KB (by entry num) total key len = 1.809527 GB avg = 0.028 KB total ukey len = 1.261239 GB avg = 0.016 KB total ukey num = 0.078718496 Billion total entry num = 0.065526225 Billion write speed all = 0.538346659 MB/sec (with seq num) write speed all = 0.515308759 MB/sec (without seq num) 2024/11/23-15:14:26.497500 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265797: 930745 keys, 38992644 bytes, temperature: kWarm 2024/11/23-15:14:26.497570 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346066497534, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265797, "file_size": 38992644, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776021, "table_properties": {"data_size": 33173072, "index_size": 1781888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26060860, "raw_average_key_size": 28, "raw_value_size": 108156994, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930745, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346063, "oldest_key_time": 0, "file_creation_time": 1732346063, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265797, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:26.533604 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265798.sst) /dev/shm/ToplingTemp/Topling-89534-HTg6xl 2024/11/23-15:14:27.366737 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265798.sst): first pass time = 0.83's, 161.109'MB/sec 2024/11/23-15:14:28.680299 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265798.sst): old prealloc_size = 147639500, real_size = 32807856 2024/11/23-15:14:28.680880 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265798.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928905 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0308 GB avg-val = 116.49 avg-zval = 33.17 2024/11/23-15:14:28.683040 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265798.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:29.538957 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265798.sst): second pass time = 1.31's, 82.430'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.125'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3171.696'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 290.751'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35181.163'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.430'MB/sec zip pipeline throughput = 82.430'MB/sec entries = 928905 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.49 avg-zval = 33.17 usrkeys = 928905 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.49 avg-zval = 33.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928905 bytes = 3367296 } seq expand size = 6502335 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5133 value = 3.5123 dict = 1.00 all = 3.6247 } Zip/UnZip{ index = 0.0951 value = 0.2847 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 10.554851 GB avg = 0.159 KB (by entry num) total key len = 1.835536 GB avg = 0.028 KB total ukey len = 1.279817 GB avg = 0.016 KB total ukey num = 0.079647401 Billion total entry num = 0.066455130 Billion write speed all = 0.544457104 MB/sec (with seq num) write speed all = 0.521095788 MB/sec (without seq num) 2024/11/23-15:14:29.584563 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265798: 928905 keys, 37028350 bytes, temperature: kWarm 2024/11/23-15:14:29.584642 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346069584597, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265798, "file_size": 37028350, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776210, "table_properties": {"data_size": 30808352, "index_size": 1767104, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26009340, "raw_average_key_size": 28, "raw_value_size": 108208490, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928905, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346066, "oldest_key_time": 0, "file_creation_time": 1732346066, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265798, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:29.631814 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265799.sst) /dev/shm/ToplingTemp/Topling-89534-9w77jn 2024/11/23-15:14:30.580907 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265799.sst): first pass time = 0.95's, 141.423'MB/sec 2024/11/23-15:14:31.876992 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265799.sst): old prealloc_size = 147639500, real_size = 32478608 2024/11/23-15:14:31.877523 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265799.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932333 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.96 avg-zval = 32.68 2024/11/23-15:14:31.879971 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265799.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:32.734414 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265799.sst): second pass time = 1.30's, 83.469'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.814'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3489.357'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 289.211'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36907.859'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.469'MB/sec zip pipeline throughput = 83.469'MB/sec entries = 932333 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.96 avg-zval = 32.68 usrkeys = 932333 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.96 avg-zval = 32.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932333 bytes = 3379728 } seq expand size = 6526331 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5140 value = 3.5480 dict = 1.00 all = 3.6562 } Zip/UnZip{ index = 0.0951 value = 0.2819 dict = 1.00 all = 0.2735 } ---------------------------- total value len = 10.669490 GB avg = 0.158 KB (by entry num) total key len = 1.861641 GB avg = 0.028 KB total ukey len = 1.298464 GB avg = 0.016 KB total ukey num = 0.080579734 Billion total entry num = 0.067387463 Billion write speed all = 0.550564361 MB/sec (with seq num) write speed all = 0.526878624 MB/sec (without seq num) 2024/11/23-15:14:32.778510 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265799: 932333 keys, 36709356 bytes, temperature: kWarm 2024/11/23-15:14:32.778586 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346072778543, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265799, "file_size": 36709356, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776248, "table_properties": {"data_size": 30471808, "index_size": 1773504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26105324, "raw_average_key_size": 28, "raw_value_size": 108112498, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932333, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346069, "oldest_key_time": 0, "file_creation_time": 1732346069, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265799, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:32.813731 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265800.sst) /dev/shm/ToplingTemp/Topling-89534-rstBak 2024/11/23-15:14:33.879572 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265800.sst): first pass time = 1.07's, 125.931'MB/sec 2024/11/23-15:14:35.188572 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265800.sst): old prealloc_size = 147639500, real_size = 34112160 2024/11/23-15:14:35.189166 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265800.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933499 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0321 GB avg-val = 115.78 avg-zval = 34.36 2024/11/23-15:14:35.190014 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265800.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:36.032652 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265800.sst): second pass time = 1.31's, 82.620'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 45.569'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3231.082'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.326'MB/sec write SST data time = 0.13's, 303.916'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34789.154'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.620'MB/sec zip pipeline throughput = 82.620'MB/sec entries = 933499 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.78 avg-zval = 34.36 usrkeys = 933499 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.78 avg-zval = 34.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933499 bytes = 3500640 } seq expand size = 6534493 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.3763 value = 3.3691 dict = 1.00 all = 3.4895 } Zip/UnZip{ index = 0.0964 value = 0.2968 dict = 1.00 all = 0.2866 } ---------------------------- total value len = 10.784104 GB avg = 0.158 KB (by entry num) total key len = 1.887779 GB avg = 0.028 KB total ukey len = 1.317134 GB avg = 0.016 KB total ukey num = 0.081513233 Billion total entry num = 0.068320962 Billion write speed all = 0.556667747 MB/sec (with seq num) write speed all = 0.532657377 MB/sec (without seq num) 2024/11/23-15:14:36.078843 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265800: 933499 keys, 38463064 bytes, temperature: kWarm 2024/11/23-15:14:36.078913 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346076078877, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265800, "file_size": 38463064, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776268, "table_properties": {"data_size": 32079312, "index_size": 1799296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26137972, "raw_average_key_size": 28, "raw_value_size": 108079790, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933499, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346072, "oldest_key_time": 0, "file_creation_time": 1732346072, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265800, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:36.131427 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265801.sst) /dev/shm/ToplingTemp/Topling-89534-mOLEql 2024/11/23-15:14:37.102083 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265801.sst): first pass time = 0.97's, 138.281'MB/sec 2024/11/23-15:14:38.365821 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265801.sst): old prealloc_size = 147639500, real_size = 30472528 2024/11/23-15:14:38.366402 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265801.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928658 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0285 GB avg-val = 116.53 avg-zval = 30.64 2024/11/23-15:14:38.368238 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265801.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:39.183708 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265801.sst): second pass time = 1.26's, 85.715'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.558'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.959'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 277.154'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35087.500'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.715'MB/sec zip pipeline throughput = 85.715'MB/sec entries = 928658 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.53 avg-zval = 30.64 usrkeys = 928658 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.53 avg-zval = 30.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928658 bytes = 3482480 } seq expand size = 6500606 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.3756 value = 3.8037 dict = 1.00 all = 3.8561 } Zip/UnZip{ index = 0.0964 value = 0.2629 dict = 1.00 all = 0.2593 } ---------------------------- total value len = 10.898820 GB avg = 0.157 KB (by entry num) total key len = 1.913782 GB avg = 0.028 KB total ukey len = 1.335707 GB avg = 0.016 KB total ukey num = 0.082441891 Billion total entry num = 0.069249620 Billion write speed all = 0.562771514 MB/sec (with seq num) write speed all = 0.538438149 MB/sec (without seq num) 2024/11/23-15:14:39.227766 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265801: 928658 keys, 34806688 bytes, temperature: kWarm 2024/11/23-15:14:39.227839 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346079227801, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265801, "file_size": 34806688, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3415293032, "largest_seqno": 4012776300, "table_properties": {"data_size": 28450112, "index_size": 1790080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002424, "raw_average_key_size": 28, "raw_value_size": 108215324, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928658, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346076, "oldest_key_time": 0, "file_creation_time": 1732346076, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265801, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:39.265519 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265802.sst) /dev/shm/ToplingTemp/Topling-89534-K1T90i 2024/11/23-15:14:40.133538 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265802.sst): first pass time = 0.87's, 154.633'MB/sec 2024/11/23-15:14:41.479621 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265802.sst): old prealloc_size = 147639500, real_size = 35372160 2024/11/23-15:14:41.480158 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265802.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925281 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0334 GB avg-val = 117.06 avg-zval = 36.06 2024/11/23-15:14:41.480991 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265802.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:42.312614 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265802.sst): second pass time = 1.34's, 80.534'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.657'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3438.488'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.12's, 320.885'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35218.117'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.534'MB/sec zip pipeline throughput = 80.534'MB/sec entries = 925281 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.06 avg-zval = 36.06 usrkeys = 925281 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.06 avg-zval = 36.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925281 bytes = 3469824 } seq expand size = 6476967 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0397 GB } UnZip/Zip{ index = 10.4450 value = 3.2458 dict = 1.00 all = 3.3810 } Zip/UnZip{ index = 0.0957 value = 0.3081 dict = 1.00 all = 0.2958 } ---------------------------- total value len = 11.013607 GB avg = 0.157 KB (by entry num) total key len = 1.939690 GB avg = 0.028 KB total ukey len = 1.354213 GB avg = 0.016 KB total ukey num = 0.083367172 Billion total entry num = 0.070174901 Billion write speed all = 0.568873108 MB/sec (with seq num) write speed all = 0.544218001 MB/sec (without seq num) 2024/11/23-15:14:42.359791 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265802: 925281 keys, 39697372 bytes, temperature: kWarm 2024/11/23-15:14:42.359861 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346082359824, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265802, "file_size": 39697372, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776316, "table_properties": {"data_size": 33368944, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25907868, "raw_average_key_size": 28, "raw_value_size": 108309930, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925281, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346079, "oldest_key_time": 0, "file_creation_time": 1732346079, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265802, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:42.408358 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265803.sst) /dev/shm/ToplingTemp/Topling-89534-vOo7Yi 2024/11/23-15:14:43.364677 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265803.sst): first pass time = 0.96's, 140.354'MB/sec 2024/11/23-15:14:44.679358 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265803.sst): old prealloc_size = 147639500, real_size = 31353760 2024/11/23-15:14:44.679918 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265803.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931083 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0293 GB avg-val = 116.15 avg-zval = 31.52 2024/11/23-15:14:44.681617 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265803.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:45.515636 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265803.sst): second pass time = 1.31's, 82.354'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.687'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3293.886'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 279.241'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35331.870'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.354'MB/sec zip pipeline throughput = 82.354'MB/sec entries = 931083 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.15 avg-zval = 31.52 usrkeys = 931083 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.15 avg-zval = 31.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931083 bytes = 3491584 } seq expand size = 6517581 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5132 value = 3.6848 dict = 1.00 all = 3.7599 } Zip/UnZip{ index = 0.0951 value = 0.2714 dict = 1.00 all = 0.2660 } ---------------------------- total value len = 11.128272 GB avg = 0.157 KB (by entry num) total key len = 1.965760 GB avg = 0.028 KB total ukey len = 1.372834 GB avg = 0.016 KB total ukey num = 0.084298255 Billion total entry num = 0.071105984 Billion write speed all = 0.574972941 MB/sec (with seq num) write speed all = 0.549994222 MB/sec (without seq num) 2024/11/23-15:14:45.561597 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265803: 931083 keys, 35696960 bytes, temperature: kWarm 2024/11/23-15:14:45.561676 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346085561640, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265803, "file_size": 35696960, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012776344, "table_properties": {"data_size": 29349520, "index_size": 1771264, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26070324, "raw_average_key_size": 28, "raw_value_size": 108147522, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931083, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346082, "oldest_key_time": 0, "file_creation_time": 1732346082, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265803, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:45.597658 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265804.sst) /dev/shm/ToplingTemp/Topling-89534-dfPg6m 2024/11/23-15:14:46.533873 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265804.sst): first pass time = 0.94's, 143.368'MB/sec 2024/11/23-15:14:47.827560 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265804.sst): old prealloc_size = 147639500, real_size = 31232448 2024/11/23-15:14:47.828129 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265804.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925244 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0292 GB avg-val = 117.06 avg-zval = 31.59 2024/11/23-15:14:47.828950 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265804.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:48.651750 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265804.sst): second pass time = 1.29's, 83.778'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.055'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3231.691'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 272.702'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34061.893'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.778'MB/sec zip pipeline throughput = 83.778'MB/sec entries = 925244 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.06 avg-zval = 31.59 usrkeys = 925244 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.84 avg-zval = 31.59 TagRS{ kinds = RS_Key0_TagN bytes = 144680 } TagArray{ size = 895975 bytes = 3247920 } seq expand size = 6271825 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4435 value = 3.7056 dict = 1.00 all = 3.7825 } Zip/UnZip{ index = 0.0958 value = 0.2699 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 11.242855 GB avg = 0.156 KB (by entry num) total key len = 1.991667 GB avg = 0.028 KB total ukey len = 1.391339 GB avg = 0.016 KB total ukey num = 0.085223499 Billion total entry num = 0.072031228 Billion write speed all = 0.581061972 MB/sec (with seq num) write speed all = 0.555761710 MB/sec (without seq num) 2024/11/23-15:14:48.695380 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265804: 925244 keys, 35484132 bytes, temperature: kWarm 2024/11/23-15:14:48.695449 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346088695413, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265804, "file_size": 35484132, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 29229040, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25906832, "raw_average_key_size": 28, "raw_value_size": 108310964, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925244, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346085, "oldest_key_time": 0, "file_creation_time": 1732346085, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265804, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:48.734928 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265805.sst) /dev/shm/ToplingTemp/Topling-89534-dyMC3l 2024/11/23-15:14:49.625051 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265805.sst): first pass time = 0.89's, 150.794'MB/sec 2024/11/23-15:14:51.129473 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265805.sst): old prealloc_size = 147639500, real_size = 36676784 2024/11/23-15:14:51.130103 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265805.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926738 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1083 GB zip-val = 0.0346 GB avg-val = 116.83 avg-zval = 37.39 2024/11/23-15:14:51.130255 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265805.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:51.829958 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265805.sst): second pass time = 1.50's, 72.009'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.70's, 53.914'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2970.797'MB/sec (index lex order gen) rebuild zvType time = 0.58's, 0.402'MB/sec write SST data time = 0.12's, 307.157'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 37705.462'MB/sec zip my value time = 1.50's, unzip length = 0.108'GB zip my value throughput = 72.009'MB/sec zip pipeline throughput = 72.009'MB/sec entries = 926738 avg-key = 28.00 avg-zkey = 1.94 avg-val = 116.83 avg-zval = 37.39 usrkeys = 926738 avg-key = 20.00 avg-zkey = 1.94 avg-val = 117.05 avg-zval = 37.39 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 29019 bytes = 83440 } seq expand size = 203133 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.3099 value = 3.1249 dict = 1.00 all = 3.5541 } Zip/UnZip{ index = 0.0970 value = 0.3200 dict = 1.00 all = 0.2814 } ---------------------------- total value len = 11.351327 GB avg = 0.156 KB (by entry num) total key len = 2.017616 GB avg = 0.028 KB total ukey len = 1.409874 GB avg = 0.016 KB total ukey num = 0.086150237 Billion total entry num = 0.072957966 Billion write speed all = 0.586881835 MB/sec (with seq num) write speed all = 0.561259641 MB/sec (without seq num) 2024/11/23-15:14:51.877475 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265805: 926738 keys, 37764369 bytes, temperature: kWarm 2024/11/23-15:14:51.877545 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346091877509, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265805, "file_size": 37764369, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3255890120, "table_properties": {"data_size": 34647136, "index_size": 1797760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25948664, "raw_average_key_size": 28, "raw_value_size": 108269188, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926738, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346088, "oldest_key_time": 0, "file_creation_time": 1732346088, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265805, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:51.914659 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265806.sst) /dev/shm/ToplingTemp/Topling-89534-MvHopk 2024/11/23-15:14:52.961757 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265806.sst): first pass time = 1.05's, 128.187'MB/sec 2024/11/23-15:14:54.378946 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265806.sst): old prealloc_size = 147639500, real_size = 37157264 2024/11/23-15:14:54.379520 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265806.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924400 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0352 GB avg-val = 117.19 avg-zval = 38.03 2024/11/23-15:14:54.380107 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265806.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-15:14:55.146423 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265806.sst): second pass time = 1.42's, 76.509'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.77's, 52.412'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3473.128'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.361'MB/sec write SST data time = 0.13's, 318.533'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 33728.594'MB/sec zip my value time = 1.42's, unzip length = 0.108'GB zip my value throughput = 76.509'MB/sec zip pipeline throughput = 76.509'MB/sec entries = 924400 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.19 avg-zval = 38.03 usrkeys = 924400 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.36 avg-zval = 38.03 TagRS{ kinds = RS_Key0_TagN bytes = 144520 } TagArray{ size = 549795 bytes = 2061744 } seq expand size = 3848565 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0352 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.4457 value = 3.0815 dict = 1.00 all = 3.3368 } Zip/UnZip{ index = 0.0957 value = 0.3245 dict = 1.00 all = 0.2997 } ---------------------------- total value len = 11.463510 GB avg = 0.155 KB (by entry num) total key len = 2.043499 GB avg = 0.028 KB total ukey len = 1.428362 GB avg = 0.016 KB total ukey num = 0.087074637 Billion total entry num = 0.073882366 Billion write speed all = 0.592856486 MB/sec (with seq num) write speed all = 0.566913428 MB/sec (without seq num) 2024/11/23-15:14:55.195825 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265806: 924400 keys, 40223644 bytes, temperature: kWarm 2024/11/23-15:14:55.195894 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346095195859, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265806, "file_size": 40223644, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4057727713, "table_properties": {"data_size": 35156032, "index_size": 1769920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25883200, "raw_average_key_size": 28, "raw_value_size": 108334672, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924400, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346091, "oldest_key_time": 0, "file_creation_time": 1732346091, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265806, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:55.233658 139957517301312 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265807.sst) /dev/shm/ToplingTemp/Topling-89534-cxkBUi 2024/11/23-15:14:55.328123 139957517301312 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265807.sst): first pass time = 0.09's, 232.074'MB/sec 2024/11/23-15:14:55.539946 139957517301312 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265807.sst): old prealloc_size = 147639500, real_size = 6551744 2024/11/23-15:14:55.540130 139957517301312 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265807.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 151070 prefix = 4 raw-key = 0.0024 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0177 GB zip-val = 0.0062 GB avg-val = 117.06 avg-zval = 41.19 2024/11/23-15:14:55.540287 139957517301312 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265807.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.022 GB 2024/11/23-15:14:55.671781 139957517301312 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00/265807.sst): second pass time = 0.21's, 83.757'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.13's, 54.420'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2238.100'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.342'MB/sec write SST data time = 0.02's, 335.797'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.185'MB, throughput = 6382.156'MB/sec zip my value time = 0.21's, unzip length = 0.018'GB zip my value throughput = 83.757'MB/sec zip pipeline throughput = 83.757'MB/sec entries = 151070 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.06 avg-zval = 41.19 usrkeys = 151070 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.06 avg-zval = 41.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 151070 bytes = 472112 } seq expand size = 1057490 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0030 GB value = 0.0177 GB dict = 0.19 MB all = 0.0219 GB } __ZipSize{ index = 0.0003 GB value = 0.0062 GB dict = 0.19 MB all = 0.0072 GB } UnZip/Zip{ index = 10.3963 value = 2.8416 dict = 1.00 all = 3.0548 } Zip/UnZip{ index = 0.0962 value = 0.3519 dict = 1.00 all = 0.3273 } ---------------------------- total value len = 11.482251 GB avg = 0.155 KB (by entry num) total key len = 2.047729 GB avg = 0.028 KB total ukey len = 1.431383 GB avg = 0.016 KB total ukey num = 0.087225707 Billion total entry num = 0.074033436 Billion write speed all = 0.593851050 MB/sec (with seq num) write speed all = 0.567855544 MB/sec (without seq num) 2024/11/23-15:14:55.686274 139957517301312 [compaction_job.cc:1948] [default] [JOB 28941] Generated table #265807: 151070 keys, 7173391 bytes, temperature: kWarm 2024/11/23-15:14:55.686326 139957517301312 EVENT_LOG_v1 {"time_micros": 1732346095686297, "cf_name": "default", "job": 28941, "event": "table_file_creation", "file_number": 265807, "file_size": 7173391, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4033507915, "largest_seqno": 4056215629, "table_properties": {"data_size": 6223152, "index_size": 290624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4229960, "raw_average_key_size": 28, "raw_value_size": 17683620, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 151070, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732346095, "oldest_key_time": 0, "file_creation_time": 1732346095, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265807, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-15:14:55.698922 139957517301312 [compaction_job.cc:703] job-28941: subcompact[0], size: 0.873759 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28941/att-00 [265784,265785,265786,265787,265788,265789,265790,265791,265792,265793,265794,265795,265796,265797,265798,265799,265800,265801,265802,265803,265804,265805,265806,265807] 2024/11/23-15:14:55.835404 139957517301312 (Original Log Time 2024/11/23-15:13:30.891382) EVENT_LOG_v1 {"time_micros": 1732346010891337, "job": 28941, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [265775], "files_L3": [265729, 265730, 265731, 265732, 265733, 265734, 265735, 265736, 265737, 265738, 265739, 265740, 265741, 265742, 265743, 265744, 265745, 265746, 265747, 265748, 265749, 265750, 265751], "score": 0.993712, "input_data_size": 890019654, "oldest_snapshot_seqno": 4057760996} 2024/11/23-15:14:55.835408 139957517301312 (Original Log Time 2024/11/23-15:14:55.833692) [compaction_job.cc:2022] [default] [JOB 28941] Compacted 1@2 + 23@3 files to L3 => 873758915 bytes 2024/11/23-15:14:55.835409 139957517301312 (Original Log Time 2024/11/23-15:14:55.835331) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 24 0 0 0] max score 0.03, MB/sec: 10.5 rd, 10.3 wr, level 3, files in(1, 23) out(24 +0 blob) MB in(21.4, 827.4 +0.0 blob) out(833.3 +0.0 blob), read-write-amplify(78.7) write-amplify(39.0) OK, records in: 21706946, records dropped: 31674 output_compression: Snappy 2024/11/23-15:14:55.835413 139957517301312 (Original Log Time 2024/11/23-15:14:55.835378) EVENT_LOG_v1 {"time_micros": 1732346095835356, "cf": "default", "job": 28941, "event": "compaction_finished", "compaction_time_micros": 84807545, "compaction_time_cpu_micros": 34114237, "output_level": 3, "num_output_files": 24, "total_output_size": 873758915, "num_input_records": 21706946, "num_output_records": 21675272, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 639643302, "file_range_sync_nanos": 0, "file_fsync_nanos": 18417, "file_prepare_write_nanos": 68281, "lsm_state": [0, 0, 0, 24, 0, 0, 0]} 2024/11/23-15:14:55.972858 139957517301312 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732346010,"executesMs":85094,"compactionJobId":28941,"attempt":0,"compactionInputRawBytes":3113420825,"compactionInputZipBytes":890019654,"compactionOutputRawBytes":0,"compactionOutputZipBytes":873758915,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-cdd4b164677cb4dbb41513c92a0dc75e-ebddfd3f208bf92a-00"}}, response = 2024/11/23-15:14:55.972947 139957517301312 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28941/att-00: olev 3, work 85.095 s, result 7.694 ms, install 1.737 ms, input{raw 2.900 GiB zip 848.789 MiB}