2024/11/22-19:43:12.850448 140024919680576 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00: fork to child time = 0.009760 sec 2024/11/22-19:43:12.851335 140024919680576 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00 2024/11/22-19:43:12.858510 140024919680576 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00 2024/11/22-19:43:12.859630 140024919680576 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-19:43:12.859912 140024919680576 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-19:43:12.921895 140024919680576 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09172/att-00/MANIFEST-206347 2024/11/22-19:43:12.951127 140024919680576 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09172/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 223965, last_sequence is 3512562725, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 223824 2024/11/22-19:43:12.951135 140024919680576 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-19:43:12.951235 140024919680576 [version_set.cc:5721] Creating manifest 223965 2024/11/22-19:43:12.997571 140024919680576 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-19:43:13.027765 140024919680576 [compaction_job.cc:2437] [default] [JOB 9172] Compacting 1@2 + 12@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/22-19:43:13.027795 140024919680576 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 6941 Base level 2, inputs: [223961(27MB)], [223908(35MB) 223909(36MB) 223910(36MB) 223911(36MB) 223912(37MB) 223913(37MB) 223914(35MB) 223915(36MB) 223916(35MB) 223917(36MB) 223918(35MB) 223921(35MB)] 2024/11/22-19:43:13.033703 140024919680576 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:43:13.033741 140024919680576 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:43:13.033762 140024919680576 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:43:13.033770 140024919680576 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-19:43:13.033789 140024919680576 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-19:43:13.495046 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223966.sst) /dev/shm/ToplingTemp/Topling-264005-sUp7j1 2024/11/22-19:43:14.092487 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223966.sst): first pass time = 0.60's, 224.671'MB/sec 2024/11/22-19:43:16.263177 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223966.sst): old prealloc_size = 147639500, real_size = 33565088 2024/11/22-19:43:16.263873 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223966.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923738 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.30 avg-zval = 34.18 2024/11/22-19:43:16.264712 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223966.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:17.657684 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223966.sst): second pass time = 2.17's, 49.960'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.39's, 26.920'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2589.215'MB/sec (index lex order gen) rebuild zvType time = 1.23's, 0.188'MB/sec write SST data time = 0.17's, 223.801'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 38110.802'MB/sec zip my value time = 2.17's, unzip length = 0.108'GB zip my value throughput = 49.960'MB/sec zip pipeline throughput = 129.362'MB/sec entries = 923738 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.30 avg-zval = 34.18 usrkeys = 923738 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.30 avg-zval = 34.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923738 bytes = 3117632 } seq expand size = 6466166 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5138 value = 3.4314 dict = 1.00 all = 3.5753 } Zip/UnZip{ index = 0.0951 value = 0.2914 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 0.114819 GB avg = 0.124 KB (by entry num) total key len = 0.025865 GB avg = 0.028 KB total ukey len = 0.018475 GB avg = 0.020 KB total ukey num = 0.000923738 Billion total entry num = 0.000923738 Billion write speed all = 0.003602240 MB/sec (with seq num) write speed all = 0.003413020 MB/sec (without seq num) 2024/11/22-19:43:17.693153 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223966: 923738 keys, 37540382 bytes, temperature: kWarm 2024/11/22-19:43:17.693255 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275797693190, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223966, "file_size": 37540382, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038008, "largest_seqno": 3512538861, "table_properties": {"data_size": 31576784, "index_size": 1757184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25864664, "raw_average_key_size": 28, "raw_value_size": 108353092, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923738, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275793, "oldest_key_time": 0, "file_creation_time": 1732275793, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223966, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:17.732754 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223967.sst) /dev/shm/ToplingTemp/Topling-264005-nMVhh2 2024/11/22-19:43:18.870287 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223967.sst): first pass time = 1.14's, 117.996'MB/sec 2024/11/22-19:43:21.070250 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223967.sst): old prealloc_size = 147639500, real_size = 34121808 2024/11/22-19:43:21.070942 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223967.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926584 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0321 GB avg-val = 116.85 avg-zval = 34.67 2024/11/22-19:43:21.071739 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223967.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:22.185289 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223967.sst): second pass time = 2.20's, 49.265'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 34.173'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2639.565'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.252'MB/sec write SST data time = 0.19's, 195.859'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30301.762'MB/sec zip my value time = 2.20's, unzip length = 0.108'GB zip my value throughput = 49.265'MB/sec zip pipeline throughput = 83.602'MB/sec entries = 926584 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.85 avg-zval = 34.67 usrkeys = 926584 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.85 avg-zval = 34.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926584 bytes = 3127232 } seq expand size = 6486088 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5114 value = 3.3702 dict = 1.00 all = 3.5225 } Zip/UnZip{ index = 0.0951 value = 0.2967 dict = 1.00 all = 0.2839 } ---------------------------- total value len = 0.229579 GB avg = 0.124 KB (by entry num) total key len = 0.051809 GB avg = 0.028 KB total ukey len = 0.037006 GB avg = 0.020 KB total ukey num = 0.001850322 Billion total entry num = 0.001850322 Billion write speed all = 0.007204155 MB/sec (with seq num) write speed all = 0.006825176 MB/sec (without seq num) 2024/11/22-19:43:22.222317 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223967: 926584 keys, 38103324 bytes, temperature: kWarm 2024/11/22-19:43:22.222397 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275802222352, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223967, "file_size": 38103324, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3512538877, "table_properties": {"data_size": 32126976, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25944352, "raw_average_key_size": 28, "raw_value_size": 108273384, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926584, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275797, "oldest_key_time": 0, "file_creation_time": 1732275797, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223967, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:22.292126 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223968.sst) /dev/shm/ToplingTemp/Topling-264005-myoiAZ 2024/11/22-19:43:23.425069 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223968.sst): first pass time = 1.13's, 118.475'MB/sec 2024/11/22-19:43:25.184937 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223968.sst): old prealloc_size = 147639500, real_size = 34198960 2024/11/22-19:43:25.185583 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223968.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926991 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0322 GB avg-val = 116.79 avg-zval = 34.74 2024/11/22-19:43:25.186407 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223968.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:26.341253 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223968.sst): second pass time = 1.76's, 61.570'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.16's, 33.020'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2831.559'MB/sec (index lex order gen) rebuild zvType time = 1.00's, 0.232'MB/sec write SST data time = 0.16's, 243.660'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34271.871'MB/sec zip my value time = 1.76's, unzip length = 0.108'GB zip my value throughput = 61.570'MB/sec zip pipeline throughput = 82.171'MB/sec entries = 926991 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.79 avg-zval = 34.74 usrkeys = 926991 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.79 avg-zval = 34.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926991 bytes = 3128608 } seq expand size = 6488937 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5141 value = 3.3618 dict = 1.00 all = 3.5152 } Zip/UnZip{ index = 0.0951 value = 0.2975 dict = 1.00 all = 0.2845 } ---------------------------- total value len = 0.344330 GB avg = 0.124 KB (by entry num) total key len = 0.077765 GB avg = 0.028 KB total ukey len = 0.055546 GB avg = 0.020 KB total ukey num = 0.002777313 Billion total entry num = 0.002777313 Billion write speed all = 0.010805412 MB/sec (with seq num) write speed all = 0.010236630 MB/sec (without seq num) 2024/11/22-19:43:26.375033 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223968: 926991 keys, 38181726 bytes, temperature: kWarm 2024/11/22-19:43:26.375094 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275806375061, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223968, "file_size": 38181726, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3512538911, "table_properties": {"data_size": 32203680, "index_size": 1763328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25955748, "raw_average_key_size": 28, "raw_value_size": 108262046, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926991, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275802, "oldest_key_time": 0, "file_creation_time": 1732275802, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223968, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:26.412081 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223969.sst) /dev/shm/ToplingTemp/Topling-264005-RSY8K0 2024/11/22-19:43:27.340644 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223969.sst): first pass time = 0.93's, 144.551'MB/sec 2024/11/22-19:43:28.712511 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223969.sst): old prealloc_size = 147639500, real_size = 33797888 2024/11/22-19:43:28.713103 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223969.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926737 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0318 GB avg-val = 116.83 avg-zval = 34.32 2024/11/22-19:43:28.713891 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223969.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:29.736518 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223969.sst): second pass time = 1.37's, 78.957'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.02's, 36.895'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3099.760'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.259'MB/sec write SST data time = 0.13's, 292.397'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 26769.494'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.957'MB/sec zip pipeline throughput = 78.957'MB/sec entries = 926737 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 34.32 usrkeys = 926737 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 34.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926737 bytes = 3127760 } seq expand size = 6487159 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0318 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5128 value = 3.4044 dict = 1.00 all = 3.5526 } Zip/UnZip{ index = 0.0951 value = 0.2937 dict = 1.00 all = 0.2815 } ---------------------------- total value len = 0.459086 GB avg = 0.124 KB (by entry num) total key len = 0.103713 GB avg = 0.028 KB total ukey len = 0.074081 GB avg = 0.020 KB total ukey num = 0.003704050 Billion total entry num = 0.003704050 Billion write speed all = 0.014406139 MB/sec (with seq num) write speed all = 0.013647630 MB/sec (without seq num) 2024/11/22-19:43:29.769134 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223969: 926737 keys, 37780182 bytes, temperature: kWarm 2024/11/22-19:43:29.769196 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275809769164, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223969, "file_size": 37780182, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3512538934, "table_properties": {"data_size": 31802928, "index_size": 1763072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25948636, "raw_average_key_size": 28, "raw_value_size": 108269198, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926737, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275806, "oldest_key_time": 0, "file_creation_time": 1732275806, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223969, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:29.807099 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223970.sst) /dev/shm/ToplingTemp/Topling-264005-f5q2BZ 2024/11/22-19:43:30.681891 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223970.sst): first pass time = 0.87's, 153.435'MB/sec 2024/11/22-19:43:32.065004 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223970.sst): old prealloc_size = 147639500, real_size = 34288816 2024/11/22-19:43:32.065589 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223970.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929147 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.45 avg-zval = 34.74 2024/11/22-19:43:32.066400 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223970.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:33.074714 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223970.sst): second pass time = 1.38's, 78.267'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 37.911'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3220.902'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.264'MB/sec write SST data time = 0.13's, 294.823'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32464.596'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.267'MB/sec zip pipeline throughput = 78.267'MB/sec entries = 929147 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.45 avg-zval = 34.74 usrkeys = 929147 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.45 avg-zval = 34.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929147 bytes = 3135888 } seq expand size = 6504029 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0383 GB } UnZip/Zip{ index = 10.4449 value = 3.3523 dict = 1.00 all = 3.5064 } Zip/UnZip{ index = 0.0957 value = 0.2983 dict = 1.00 all = 0.2852 } ---------------------------- total value len = 0.573792 GB avg = 0.124 KB (by entry num) total key len = 0.129730 GB avg = 0.028 KB total ukey len = 0.092664 GB avg = 0.020 KB total ukey num = 0.004633197 Billion total entry num = 0.004633197 Billion write speed all = 0.018006699 MB/sec (with seq num) write speed all = 0.017058002 MB/sec (without seq num) 2024/11/22-19:43:33.107450 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223970: 929147 keys, 38278028 bytes, temperature: kWarm 2024/11/22-19:43:33.107516 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275813107482, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223970, "file_size": 38278028, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3512538974, "table_properties": {"data_size": 32277216, "index_size": 1779136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26016116, "raw_average_key_size": 28, "raw_value_size": 108201730, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929147, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275809, "oldest_key_time": 0, "file_creation_time": 1732275809, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223970, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:33.142432 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223971.sst) /dev/shm/ToplingTemp/Topling-264005-I19ED1 2024/11/22-19:43:33.926885 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223971.sst): first pass time = 0.78's, 171.106'MB/sec 2024/11/22-19:43:35.378570 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223971.sst): old prealloc_size = 147639500, real_size = 37079584 2024/11/22-19:43:35.379174 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223971.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925064 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0351 GB avg-val = 117.09 avg-zval = 37.92 2024/11/22-19:43:35.379630 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223971.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:36.210870 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223971.sst): second pass time = 1.45's, 74.678'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 47.594'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3051.189'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 309.014'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33023.891'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.678'MB/sec zip pipeline throughput = 74.678'MB/sec entries = 925064 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.09 avg-zval = 37.92 usrkeys = 925064 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.28 avg-zval = 37.92 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 421541 bytes = 1528096 } seq expand size = 2950787 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0351 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4445 value = 3.0880 dict = 1.00 all = 3.3883 } Zip/UnZip{ index = 0.0957 value = 0.3238 dict = 1.00 all = 0.2951 } ---------------------------- total value len = 0.685059 GB avg = 0.123 KB (by entry num) total key len = 0.155631 GB avg = 0.028 KB total ukey len = 0.111165 GB avg = 0.020 KB total ukey num = 0.005558261 Billion total entry num = 0.005558261 Billion write speed all = 0.021515816 MB/sec (with seq num) write speed all = 0.020377794 MB/sec (without seq num) 2024/11/22-19:43:36.244946 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223971: 925064 keys, 39611577 bytes, temperature: kWarm 2024/11/22-19:43:36.245021 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275816244987, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223971, "file_size": 39611577, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3512537972, "table_properties": {"data_size": 35076752, "index_size": 1771392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25901792, "raw_average_key_size": 28, "raw_value_size": 108316032, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925064, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275813, "oldest_key_time": 0, "file_creation_time": 1732275813, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223971, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:36.284753 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223972.sst) /dev/shm/ToplingTemp/Topling-264005-7yjqr1 2024/11/22-19:43:37.095451 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223972.sst): first pass time = 0.81's, 165.568'MB/sec 2024/11/22-19:43:38.718749 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223972.sst): old prealloc_size = 147639500, real_size = 36499104 2024/11/22-19:43:38.719336 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223972.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927106 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.77 avg-zval = 37.20 2024/11/22-19:43:38.719379 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223972.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:39.483112 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223972.sst): second pass time = 1.62's, 66.717'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 48.875'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3184.754'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.364'MB/sec write SST data time = 0.13's, 294.688'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33030.110'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.717'MB/sec zip pipeline throughput = 66.717'MB/sec entries = 927106 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.77 avg-zval = 37.20 usrkeys = 927106 avg-key = 20.00 avg-zkey = 1.91 avg-val = 116.77 avg-zval = 37.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4441 value = 3.1387 dict = 1.00 all = 3.5928 } Zip/UnZip{ index = 0.0957 value = 0.3186 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 0.793318 GB avg = 0.122 KB (by entry num) total key len = 0.181590 GB avg = 0.028 KB total ukey len = 0.129707 GB avg = 0.020 KB total ukey num = 0.006485367 Billion total entry num = 0.006485367 Billion write speed all = 0.024948770 MB/sec (with seq num) write speed all = 0.023621039 MB/sec (without seq num) 2024/11/22-19:43:39.518188 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223972: 927106 keys, 37357380 bytes, temperature: kWarm 2024/11/22-19:43:39.518251 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275819518219, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223972, "file_size": 37357380, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34491792, "index_size": 1775360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25958968, "raw_average_key_size": 28, "raw_value_size": 108258872, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927106, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275816, "oldest_key_time": 0, "file_creation_time": 1732275816, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223972, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:39.554975 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223973.sst) /dev/shm/ToplingTemp/Topling-264005-ErgSY1 2024/11/22-19:43:40.388163 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223973.sst): first pass time = 0.83's, 161.098'MB/sec 2024/11/22-19:43:41.941493 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223973.sst): old prealloc_size = 147639500, real_size = 36867136 2024/11/22-19:43:41.942083 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223973.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925485 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0349 GB avg-val = 117.02 avg-zval = 37.68 2024/11/22-19:43:41.942328 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223973.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:42.623520 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223973.sst): second pass time = 1.55's, 69.753'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.68's, 56.378'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3086.863'MB/sec (index lex order gen) rebuild zvType time = 0.56's, 0.417'MB/sec write SST data time = 0.13's, 304.681'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27439.896'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.753'MB/sec zip pipeline throughput = 88.128'MB/sec entries = 925485 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.02 avg-zval = 37.68 usrkeys = 925485 avg-key = 20.00 avg-zkey = 1.90 avg-val = 118.37 avg-zval = 37.68 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 178343 bytes = 579632 } seq expand size = 1248401 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0349 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.5123 value = 3.1055 dict = 1.00 all = 3.4907 } Zip/UnZip{ index = 0.0951 value = 0.3220 dict = 1.00 all = 0.2865 } ---------------------------- total value len = 0.902870 GB avg = 0.122 KB (by entry num) total key len = 0.207504 GB avg = 0.028 KB total ukey len = 0.148217 GB avg = 0.020 KB total ukey num = 0.007410852 Billion total entry num = 0.007410852 Billion write speed all = 0.028413190 MB/sec (with seq num) write speed all = 0.026896109 MB/sec (without seq num) 2024/11/22-19:43:42.656474 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223973: 925485 keys, 38450317 bytes, temperature: kWarm 2024/11/22-19:43:42.656537 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275822656506, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223973, "file_size": 38450317, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3310385348, "table_properties": {"data_size": 34874800, "index_size": 1760768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25913580, "raw_average_key_size": 28, "raw_value_size": 108304266, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925485, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275819, "oldest_key_time": 0, "file_creation_time": 1732275819, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223973, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:42.696118 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223974.sst) /dev/shm/ToplingTemp/Topling-264005-hvU5p2 2024/11/22-19:43:43.563075 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223974.sst): first pass time = 0.87's, 154.822'MB/sec 2024/11/22-19:43:45.040195 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223974.sst): old prealloc_size = 147639500, real_size = 32238240 2024/11/22-19:43:45.040768 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223974.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923154 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0303 GB avg-val = 117.39 avg-zval = 32.77 2024/11/22-19:43:45.042719 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223974.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:45.923422 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223974.sst): second pass time = 1.48's, 73.422'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.131'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3116.185'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.306'MB/sec write SST data time = 0.13's, 281.549'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 29787.276'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.422'MB/sec zip pipeline throughput = 75.311'MB/sec entries = 923154 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 32.77 usrkeys = 923154 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 32.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923154 bytes = 3231056 } seq expand size = 6462078 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5126 value = 3.5823 dict = 1.00 all = 3.6947 } Zip/UnZip{ index = 0.0951 value = 0.2791 dict = 1.00 all = 0.2707 } ---------------------------- total value len = 1.017702 GB avg = 0.122 KB (by entry num) total key len = 0.233352 GB avg = 0.028 KB total ukey len = 0.166680 GB avg = 0.020 KB total ukey num = 0.008334006 Billion total entry num = 0.008334006 Billion write speed all = 0.032010323 MB/sec (with seq num) write speed all = 0.030304406 MB/sec (without seq num) 2024/11/22-19:43:45.954468 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223974: 923154 keys, 36327280 bytes, temperature: kWarm 2024/11/22-19:43:45.954531 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275825954498, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223974, "file_size": 36327280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3310385349, "largest_seqno": 3512538449, "table_properties": {"data_size": 30250960, "index_size": 1756288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25848312, "raw_average_key_size": 28, "raw_value_size": 108369500, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923154, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275822, "oldest_key_time": 0, "file_creation_time": 1732275822, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223974, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:45.993077 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223975.sst) /dev/shm/ToplingTemp/Topling-264005-Ke62d3 2024/11/22-19:43:46.907217 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223975.sst): first pass time = 0.91's, 146.830'MB/sec 2024/11/22-19:43:48.287864 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223975.sst): old prealloc_size = 147639500, real_size = 34625056 2024/11/22-19:43:48.288438 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223975.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927055 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 116.78 avg-zval = 35.18 2024/11/22-19:43:48.289253 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223975.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:49.200339 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223975.sst): second pass time = 1.38's, 78.471'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 42.439'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3217.836'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 303.431'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30952.149'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.471'MB/sec zip pipeline throughput = 78.471'MB/sec entries = 927055 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.78 avg-zval = 35.18 usrkeys = 927055 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.78 avg-zval = 35.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927055 bytes = 3244704 } seq expand size = 6489385 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.4409 value = 3.3191 dict = 1.00 all = 3.4660 } Zip/UnZip{ index = 0.0958 value = 0.3013 dict = 1.00 all = 0.2885 } ---------------------------- total value len = 1.132451 GB avg = 0.122 KB (by entry num) total key len = 0.259310 GB avg = 0.028 KB total ukey len = 0.185221 GB avg = 0.020 KB total ukey num = 0.009261061 Billion total entry num = 0.009261061 Billion write speed all = 0.035607566 MB/sec (with seq num) write speed all = 0.033712047 MB/sec (without seq num) 2024/11/22-19:43:49.236726 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223975: 927055 keys, 38724022 bytes, temperature: kWarm 2024/11/22-19:43:49.236803 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275829236756, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223975, "file_size": 38724022, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3512538303, "table_properties": {"data_size": 32617296, "index_size": 1775808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957540, "raw_average_key_size": 28, "raw_value_size": 108260198, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927055, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275825, "oldest_key_time": 0, "file_creation_time": 1732275825, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223975, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:49.279971 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223976.sst) /dev/shm/ToplingTemp/Topling-264005-iVD0v2 2024/11/22-19:43:50.535651 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223976.sst): first pass time = 1.26's, 106.892'MB/sec 2024/11/22-19:43:52.147333 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223976.sst): old prealloc_size = 147639500, real_size = 33731120 2024/11/22-19:43:52.147937 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223976.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923439 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.35 avg-zval = 34.37 2024/11/22-19:43:52.148756 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223976.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:53.098953 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223976.sst): second pass time = 1.61's, 67.263'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.745'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3023.006'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.280'MB/sec write SST data time = 0.13's, 295.651'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34255.432'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.263'MB/sec zip pipeline throughput = 67.263'MB/sec entries = 923439 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.35 avg-zval = 34.37 usrkeys = 923439 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.35 avg-zval = 34.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923439 bytes = 3232048 } seq expand size = 6464073 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5112 value = 3.4137 dict = 1.00 all = 3.5487 } Zip/UnZip{ index = 0.0951 value = 0.2929 dict = 1.00 all = 0.2818 } ---------------------------- total value len = 1.247277 GB avg = 0.122 KB (by entry num) total key len = 0.285166 GB avg = 0.028 KB total ukey len = 0.203690 GB avg = 0.020 KB total ukey num = 0.010184500 Billion total entry num = 0.010184500 Billion write speed all = 0.039202939 MB/sec (with seq num) write speed all = 0.037118621 MB/sec (without seq num) 2024/11/22-19:43:53.132855 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223976: 923439 keys, 37821182 bytes, temperature: kWarm 2024/11/22-19:43:53.132922 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275833132885, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223976, "file_size": 37821182, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3378289821, "largest_seqno": 3512538390, "table_properties": {"data_size": 31743008, "index_size": 1757056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25856292, "raw_average_key_size": 28, "raw_value_size": 108361562, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923439, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275829, "oldest_key_time": 0, "file_creation_time": 1732275829, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223976, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:53.170341 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223977.sst) /dev/shm/ToplingTemp/Topling-264005-r2DrdZ 2024/11/22-19:43:54.214074 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223977.sst): first pass time = 1.04's, 128.600'MB/sec 2024/11/22-19:43:55.996742 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223977.sst): old prealloc_size = 147639500, real_size = 33602336 2024/11/22-19:43:55.997292 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223977.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923393 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.35 avg-zval = 34.24 2024/11/22-19:43:55.998115 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223977.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:43:56.947443 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223977.sst): second pass time = 1.78's, 60.808'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.647'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3339.103'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 288.538'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36378.460'MB/sec zip my value time = 1.78's, unzip length = 0.108'GB zip my value throughput = 60.808'MB/sec zip pipeline throughput = 60.808'MB/sec entries = 923393 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.35 avg-zval = 34.24 usrkeys = 923393 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.35 avg-zval = 34.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923393 bytes = 3231888 } seq expand size = 6463751 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5130 value = 3.4276 dict = 1.00 all = 3.5609 } Zip/UnZip{ index = 0.0951 value = 0.2917 dict = 1.00 all = 0.2808 } ---------------------------- total value len = 1.362104 GB avg = 0.123 KB (by entry num) total key len = 0.311021 GB avg = 0.028 KB total ukey len = 0.222158 GB avg = 0.020 KB total ukey num = 0.011107893 Billion total entry num = 0.011107893 Billion write speed all = 0.042797640 MB/sec (with seq num) write speed all = 0.040524568 MB/sec (without seq num) 2024/11/22-19:43:56.982728 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223977: 923393 keys, 37692082 bytes, temperature: kWarm 2024/11/22-19:43:56.982800 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275836982759, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223977, "file_size": 37692082, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774735, "largest_seqno": 3512538122, "table_properties": {"data_size": 31614608, "index_size": 1756672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25855004, "raw_average_key_size": 28, "raw_value_size": 108362854, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923393, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275833, "oldest_key_time": 0, "file_creation_time": 1732275833, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223977, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:57.021739 140024919680576 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223978.sst) /dev/shm/ToplingTemp/Topling-264005-mL4hw1 2024/11/22-19:43:57.116656 140024919680576 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223978.sst): first pass time = 0.09's, 219.147'MB/sec 2024/11/22-19:43:57.304799 140024919680576 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223978.sst): old prealloc_size = 147639500, real_size = 6055024 2024/11/22-19:43:57.304936 140024919680576 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223978.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 143171 prefix = 4 raw-key = 0.0023 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0168 GB zip-val = 0.0057 GB avg-val = 117.20 avg-zval = 40.12 2024/11/22-19:43:57.305076 140024919680576 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223978.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.021 GB 2024/11/22-19:43:57.440000 140024919680576 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00/223978.sst): second pass time = 0.19's, 89.452'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.14's, 49.147'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2473.413'MB/sec (index lex order gen) rebuild zvType time = 0.11's, 0.312'MB/sec write SST data time = 0.02's, 325.538'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.176'MB, throughput = 5595.494'MB/sec zip my value time = 0.19's, unzip length = 0.017'GB zip my value throughput = 89.452'MB/sec zip pipeline throughput = 89.452'MB/sec entries = 143171 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.20 avg-zval = 40.12 usrkeys = 143171 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.20 avg-zval = 40.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 143171 bytes = 447424 } seq expand size = 1002197 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0029 GB value = 0.0168 GB dict = 0.18 MB all = 0.0208 GB } __ZipSize{ index = 0.0003 GB value = 0.0057 GB dict = 0.18 MB all = 0.0066 GB } UnZip/Zip{ index = 10.4000 value = 2.9214 dict = 1.00 all = 3.1286 } Zip/UnZip{ index = 0.0962 value = 0.3423 dict = 1.00 all = 0.3196 } ---------------------------- total value len = 1.379885 GB avg = 0.123 KB (by entry num) total key len = 0.315030 GB avg = 0.028 KB total ukey len = 0.225021 GB avg = 0.020 KB total ukey num = 0.011251064 Billion total entry num = 0.011251064 Billion write speed all = 0.043354484 MB/sec (with seq num) write speed all = 0.041052143 MB/sec (without seq num) 2024/11/22-19:43:57.451321 140024919680576 [compaction_job.cc:1948] [default] [JOB 9172] Generated table #223978: 143171 keys, 6644517 bytes, temperature: kWarm 2024/11/22-19:43:57.451380 140024919680576 EVENT_LOG_v1 {"time_micros": 1732275837451347, "cf_name": "default", "job": 9172, "event": "table_file_creation", "file_number": 223978, "file_size": 6644517, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3483939867, "largest_seqno": 3511085533, "table_properties": {"data_size": 5743712, "index_size": 275328, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4008788, "raw_average_key_size": 28, "raw_value_size": 16779550, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 143171, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732275837, "oldest_key_time": 0, "file_creation_time": 1732275837, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 223978, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:43:57.461434 140024919680576 [compaction_job.cc:703] job-09172: subcompact[0], size: 0.462512 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-09172/att-00 [223966,223967,223968,223969,223970,223971,223972,223973,223974,223975,223976,223977,223978] 2024/11/22-19:43:57.546400 140024919680576 (Original Log Time 2024/11/22-19:43:13.027862) EVENT_LOG_v1 {"time_micros": 1732275793027811, "job": 9172, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [223961], "files_L3": [223908, 223909, 223910, 223911, 223912, 223913, 223914, 223915, 223916, 223917, 223918, 223921], "score": 0.991353, "input_data_size": 484688474, "oldest_snapshot_seqno": 3512562697} 2024/11/22-19:43:57.546403 140024919680576 (Original Log Time 2024/11/22-19:43:57.545708) [compaction_job.cc:2022] [default] [JOB 9172] Compacted 1@2 + 12@3 files to L3 => 462511999 bytes 2024/11/22-19:43:57.546405 140024919680576 (Original Log Time 2024/11/22-19:43:57.546333) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 10.9 rd, 10.4 wr, level 3, files in(1, 12) out(13 +0 blob) MB in(27.1, 435.1 +0.0 blob) out(441.1 +0.0 blob), read-write-amplify(33.3) write-amplify(16.3) OK, records in: 11340666, records dropped: 89602 output_compression: Snappy 2024/11/22-19:43:57.546411 140024919680576 (Original Log Time 2024/11/22-19:43:57.546379) EVENT_LOG_v1 {"time_micros": 1732275837546358, "cf": "default", "job": 9172, "event": "compaction_finished", "compaction_time_micros": 44433579, "compaction_time_cpu_micros": 19403415, "output_level": 3, "num_output_files": 13, "total_output_size": 462511999, "num_input_records": 11340666, "num_output_records": 11251064, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 315220395, "file_range_sync_nanos": 0, "file_fsync_nanos": 12630, "file_prepare_write_nanos": 29437, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/22-19:43:57.670944 140024919680576 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732275792,"executesMs":44690,"compactionJobId":9172,"attempt":0,"compactionInputRawBytes":1644125560,"compactionInputZipBytes":484688474,"compactionOutputRawBytes":0,"compactionOutputZipBytes":462511999,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-6f839cf5a9e882d5af32d842dbfe9904-82847ead919c8479-00"}}, response = 2024/11/22-19:43:57.671015 140024919680576 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-09172/att-00: olev 3, work 44.690 s, result 4.947 ms, install 0.723 ms, input{raw 1.531 GiB zip 462.235 MiB}