2024/11/23-21:06:24.390907 139957282305600 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00: fork to child time = 0.025913 sec 2024/11/23-21:06:24.391624 139957282305600 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00 2024/11/23-21:06:24.398407 139957282305600 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00 2024/11/23-21:06:24.399379 139957282305600 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-21:06:24.399616 139957282305600 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-21:06:24.452663 139957282305600 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35786/att-00/MANIFEST-206347 2024/11/23-21:06:24.489826 139957282305600 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35786/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 280654, last_sequence is 4262388954, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 280534 2024/11/23-21:06:24.489833 139957282305600 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-21:06:24.490025 139957282305600 [version_set.cc:5721] Creating manifest 280654 2024/11/23-21:06:24.528104 139957282305600 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-21:06:24.547568 139957282305600 [compaction_job.cc:2437] [default] [JOB 35786] Compacting 1@2 + 12@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-21:06:24.547580 139957282305600 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 26785 Base level 2, inputs: [280587(27MB)], [280538(38MB) 280539(38MB) 280540(35MB) 280541(35MB) 280542(36MB) 280543(33MB) 280544(37MB) 280545(33MB) 280546(37MB) 280547(37MB) 280548(36MB) 280601(36MB)] 2024/11/23-21:06:24.558581 139957282305600 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:06:24.558604 139957282305600 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:06:24.558619 139957282305600 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-21:06:24.558627 139957282305600 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:06:24.558636 139957282305600 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-21:06:25.180593 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280655.sst) /dev/shm/ToplingTemp/Topling-188733-sdQO9g 2024/11/23-21:06:25.732508 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280655.sst): first pass time = 0.55's, 243.204'MB/sec 2024/11/23-21:06:28.805147 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280655.sst): old prealloc_size = 147639500, real_size = 37019872 2024/11/23-21:06:28.805717 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280655.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 935438 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0350 GB avg-val = 115.48 avg-zval = 37.40 2024/11/23-21:06:28.806421 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280655.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:06:29.817086 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280655.sst): second pass time = 3.07's, 35.166'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 39.900'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3254.433'MB/sec (index lex order gen) rebuild zvType time = 0.86's, 0.272'MB/sec write SST data time = 0.15's, 268.437'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 27827.372'MB/sec zip my value time = 3.07's, unzip length = 0.108'GB zip my value throughput = 35.166'MB/sec zip pipeline throughput = 136.113'MB/sec entries = 935438 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.48 avg-zval = 37.40 usrkeys = 935438 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.19 avg-zval = 37.40 TagRS{ kinds = RS_Key0_TagN bytes = 146280 } TagArray{ size = 628646 bytes = 2357440 } seq expand size = 4400522 multi value expand size = 0 cnt WriteAppend = 23 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0350 GB dict = 1.08 MB all = 0.0404 GB } UnZip/Zip{ index = 10.3764 value = 3.0880 dict = 1.00 all = 3.3241 } Zip/UnZip{ index = 0.0964 value = 0.3238 dict = 1.00 all = 0.3008 } ---------------------------- total value len = 8.961046 GB avg = 0.168 KB (by entry num) total key len = 1.467013 GB avg = 0.028 KB total ukey len = 1.016587 GB avg = 0.015 KB total ukey num = 0.066485873 Billion total entry num = 0.053293602 Billion write speed all = 0.237662466 MB/sec (with seq num) write speed all = 0.227945691 MB/sec (without seq num) 2024/11/23-21:06:29.868549 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280655: 935438 keys, 40376728 bytes, temperature: kWarm 2024/11/23-21:06:29.868643 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367189868586, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280655, "file_size": 40376728, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4261176400, "table_properties": {"data_size": 34982800, "index_size": 1803008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26192264, "raw_average_key_size": 28, "raw_value_size": 108025544, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 935438, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367185, "oldest_key_time": 0, "file_creation_time": 1732367185, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280655, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:29.915203 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280656.sst) /dev/shm/ToplingTemp/Topling-188733-MwLF3g 2024/11/23-21:06:31.141450 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280656.sst): first pass time = 1.23's, 109.458'MB/sec 2024/11/23-21:06:32.876313 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280656.sst): old prealloc_size = 147639500, real_size = 35357824 2024/11/23-21:06:32.876840 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280656.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926843 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0334 GB avg-val = 116.81 avg-zval = 35.98 2024/11/23-21:06:32.877619 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280656.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:06:33.929107 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280656.sst): second pass time = 1.73's, 62.437'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 37.480'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3511.794'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.252'MB/sec write SST data time = 0.13's, 297.848'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 39614.654'MB/sec zip my value time = 1.73's, unzip length = 0.108'GB zip my value throughput = 62.437'MB/sec zip pipeline throughput = 62.437'MB/sec entries = 926843 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.81 avg-zval = 35.98 usrkeys = 926843 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.05 avg-zval = 35.98 TagRS{ kinds = RS_Key0_TagN bytes = 144920 } TagArray{ size = 825809 bytes = 3096800 } seq expand size = 5780663 multi value expand size = 0 cnt WriteAppend = 20 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.4431 value = 3.2463 dict = 1.00 all = 3.4016 } Zip/UnZip{ index = 0.0958 value = 0.3080 dict = 1.00 all = 0.2940 } ---------------------------- total value len = 9.075093 GB avg = 0.167 KB (by entry num) total key len = 1.492965 GB avg = 0.028 KB total ukey len = 1.035124 GB avg = 0.015 KB total ukey num = 0.067412716 Billion total entry num = 0.054220445 Billion write speed all = 0.240830555 MB/sec (with seq num) write speed all = 0.230945718 MB/sec (without seq num) 2024/11/23-21:06:33.979723 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280656: 926843 keys, 39457676 bytes, temperature: kWarm 2024/11/23-21:06:33.979792 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367193979756, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280656, "file_size": 39457676, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012775967, "table_properties": {"data_size": 33350896, "index_size": 1775040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951604, "raw_average_key_size": 28, "raw_value_size": 108266158, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926843, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367189, "oldest_key_time": 0, "file_creation_time": 1732367189, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280656, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:34.019021 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280657.sst) /dev/shm/ToplingTemp/Topling-188733-TEBuEk 2024/11/23-21:06:34.944218 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280657.sst): first pass time = 0.93's, 145.077'MB/sec 2024/11/23-21:06:36.781675 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280657.sst): old prealloc_size = 147639500, real_size = 32676496 2024/11/23-21:06:36.782216 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280657.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929533 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.39 avg-zval = 33.01 2024/11/23-21:06:36.783005 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280657.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:06:37.825736 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280657.sst): second pass time = 1.84's, 58.905'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 35.302'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3377.183'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.255'MB/sec write SST data time = 0.13's, 281.479'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34681.060'MB/sec zip my value time = 1.84's, unzip length = 0.108'GB zip my value throughput = 58.905'MB/sec zip pipeline throughput = 58.905'MB/sec entries = 929533 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.39 avg-zval = 33.01 usrkeys = 929533 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.78 avg-zval = 33.01 TagRS{ kinds = RS_Key0_TagN bytes = 145320 } TagArray{ size = 847570 bytes = 3178400 } seq expand size = 5932990 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5837 value = 3.5256 dict = 1.00 all = 3.6416 } Zip/UnZip{ index = 0.0945 value = 0.2836 dict = 1.00 all = 0.2746 } ---------------------------- total value len = 9.189217 GB avg = 0.167 KB (by entry num) total key len = 1.518992 GB avg = 0.028 KB total ukey len = 1.053714 GB avg = 0.015 KB total ukey num = 0.068342249 Billion total entry num = 0.055149978 Billion write speed all = 0.244002718 MB/sec (with seq num) write speed all = 0.233949313 MB/sec (without seq num) 2024/11/23-21:06:37.871454 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280657: 929533 keys, 36856808 bytes, temperature: kWarm 2024/11/23-21:06:37.871522 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367197871487, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280657, "file_size": 36856808, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 30687360, "index_size": 1756544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26026924, "raw_average_key_size": 28, "raw_value_size": 108190846, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929533, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367194, "oldest_key_time": 0, "file_creation_time": 1732367194, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280657, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:37.912307 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280658.sst) /dev/shm/ToplingTemp/Topling-188733-k1GYZk 2024/11/23-21:06:38.994691 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280658.sst): first pass time = 1.08's, 124.006'MB/sec 2024/11/23-21:06:40.393593 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280658.sst): old prealloc_size = 147639500, real_size = 31734288 2024/11/23-21:06:40.394184 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280658.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931692 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0297 GB avg-val = 116.06 avg-zval = 31.91 2024/11/23-21:06:40.395040 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280658.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:06:41.503074 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280658.sst): second pass time = 1.40's, 77.377'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 32.415'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3121.805'MB/sec (index lex order gen) rebuild zvType time = 0.97's, 0.240'MB/sec write SST data time = 0.14's, 261.238'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34771.027'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.377'MB/sec zip pipeline throughput = 77.377'MB/sec entries = 931692 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.06 avg-zval = 31.91 usrkeys = 931692 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.06 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931692 bytes = 3377392 } seq expand size = 6521844 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5136 value = 3.6372 dict = 1.00 all = 3.7321 } Zip/UnZip{ index = 0.0951 value = 0.2749 dict = 1.00 all = 0.2679 } ---------------------------- total value len = 9.303869 GB avg = 0.166 KB (by entry num) total key len = 1.545079 GB avg = 0.028 KB total ukey len = 1.072348 GB avg = 0.015 KB total ukey num = 0.069273941 Billion total entry num = 0.056081670 Billion write speed all = 0.247188970 MB/sec (with seq num) write speed all = 0.236966582 MB/sec (without seq num) 2024/11/23-21:06:41.549162 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280658: 931692 keys, 35962740 bytes, temperature: kWarm 2024/11/23-21:06:41.549231 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367201549196, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280658, "file_size": 35962740, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776228, "table_properties": {"data_size": 29728832, "index_size": 1772352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26087376, "raw_average_key_size": 28, "raw_value_size": 108130408, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931692, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367197, "oldest_key_time": 0, "file_creation_time": 1732367197, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280658, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:41.588072 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280659.sst) /dev/shm/ToplingTemp/Topling-188733-1wrxnj 2024/11/23-21:06:42.822934 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280659.sst): first pass time = 1.23's, 108.694'MB/sec 2024/11/23-21:06:44.200345 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280659.sst): old prealloc_size = 147639500, real_size = 33350656 2024/11/23-21:06:44.201048 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280659.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933210 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0313 GB avg-val = 115.82 avg-zval = 33.57 2024/11/23-21:06:44.201937 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280659.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:06:45.261106 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280659.sst): second pass time = 1.38's, 78.530'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 35.432'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2625.168'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.257'MB/sec write SST data time = 0.15's, 245.559'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32777.889'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.530'MB/sec zip pipeline throughput = 78.530'MB/sec entries = 933210 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.82 avg-zval = 33.57 usrkeys = 933210 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.82 avg-zval = 33.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933210 bytes = 3382896 } seq expand size = 6532470 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4448 value = 3.4500 dict = 1.00 all = 3.5711 } Zip/UnZip{ index = 0.0957 value = 0.2899 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 9.418489 GB avg = 0.165 KB (by entry num) total key len = 1.571209 GB avg = 0.028 KB total ukey len = 1.091012 GB avg = 0.016 KB total ukey num = 0.070207151 Billion total entry num = 0.057014880 Billion write speed all = 0.250374469 MB/sec (with seq num) write speed all = 0.239982868 MB/sec (without seq num) 2024/11/23-21:06:45.307314 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280659: 933210 keys, 37584096 bytes, temperature: kWarm 2024/11/23-21:06:45.307387 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367205307348, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280659, "file_size": 37584096, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3485447251, "largest_seqno": 4012776268, "table_properties": {"data_size": 31330224, "index_size": 1786944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26129880, "raw_average_key_size": 28, "raw_value_size": 108087848, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933210, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367201, "oldest_key_time": 0, "file_creation_time": 1732367201, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280659, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:45.343313 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280660.sst) /dev/shm/ToplingTemp/Topling-188733-r1BXWi 2024/11/23-21:06:46.535429 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280660.sst): first pass time = 1.19's, 112.591'MB/sec 2024/11/23-21:06:47.998589 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280660.sst): old prealloc_size = 147639500, real_size = 32643792 2024/11/23-21:06:47.999320 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280660.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924918 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 117.11 avg-zval = 33.13 2024/11/23-21:06:48.000219 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280660.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:06:48.975040 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280660.sst): second pass time = 1.46's, 74.114'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 37.860'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2498.091'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.16's, 228.571'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33389.072'MB/sec zip my value time = 1.46's, unzip length = 0.108'GB zip my value throughput = 74.114'MB/sec zip pipeline throughput = 74.114'MB/sec entries = 924918 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.11 avg-zval = 33.13 usrkeys = 924918 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.11 avg-zval = 33.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924918 bytes = 3468464 } seq expand size = 6474426 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4425 value = 3.5352 dict = 1.00 all = 3.6307 } Zip/UnZip{ index = 0.0958 value = 0.2829 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 9.533284 GB avg = 0.165 KB (by entry num) total key len = 1.597107 GB avg = 0.028 KB total ukey len = 1.109511 GB avg = 0.016 KB total ukey num = 0.071132069 Billion total entry num = 0.057939798 Billion write speed all = 0.253558358 MB/sec (with seq num) write speed all = 0.242999074 MB/sec (without seq num) 2024/11/23-21:06:49.022826 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280660: 924918 keys, 36967902 bytes, temperature: kWarm 2024/11/23-21:06:49.022898 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367209022864, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280660, "file_size": 36967902, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012776300, "table_properties": {"data_size": 30640896, "index_size": 1771456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897704, "raw_average_key_size": 28, "raw_value_size": 108320164, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924918, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367205, "oldest_key_time": 0, "file_creation_time": 1732367205, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280660, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:49.059146 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280661.sst) /dev/shm/ToplingTemp/Topling-188733-4Zg4Ek 2024/11/23-21:06:50.208256 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280661.sst): first pass time = 1.15's, 116.806'MB/sec 2024/11/23-21:06:51.736902 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280661.sst): old prealloc_size = 147639500, real_size = 34461920 2024/11/23-21:06:51.737508 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280661.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929106 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0325 GB avg-val = 116.46 avg-zval = 34.94 2024/11/23-21:06:51.738326 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280661.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:06:52.610708 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280661.sst): second pass time = 1.52's, 71.100'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 44.270'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3015.694'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.13's, 308.393'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27445.213'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.100'MB/sec zip pipeline throughput = 71.100'MB/sec entries = 929106 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.46 avg-zval = 34.94 usrkeys = 929106 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.46 avg-zval = 34.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929106 bytes = 3368032 } seq expand size = 6503742 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.5144 value = 3.3332 dict = 1.00 all = 3.4697 } Zip/UnZip{ index = 0.0951 value = 0.3000 dict = 1.00 all = 0.2882 } ---------------------------- total value len = 9.647990 GB avg = 0.164 KB (by entry num) total key len = 1.623122 GB avg = 0.028 KB total ukey len = 1.128093 GB avg = 0.016 KB total ukey num = 0.072061175 Billion total entry num = 0.058868904 Billion write speed all = 0.256742834 MB/sec (with seq num) write speed all = 0.246015113 MB/sec (without seq num) 2024/11/23-21:06:52.657218 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280661: 929106 keys, 38682508 bytes, temperature: kWarm 2024/11/23-21:06:52.657292 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367212657257, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280661, "file_size": 38682508, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 32462160, "index_size": 1767296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26014968, "raw_average_key_size": 28, "raw_value_size": 108202896, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929106, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367209, "oldest_key_time": 0, "file_creation_time": 1732367209, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280661, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:52.694574 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280662.sst) /dev/shm/ToplingTemp/Topling-188733-haONCh 2024/11/23-21:06:53.728827 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280662.sst): first pass time = 1.03's, 129.778'MB/sec 2024/11/23-21:06:55.222159 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280662.sst): old prealloc_size = 147639500, real_size = 31075040 2024/11/23-21:06:55.222704 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280662.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929948 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0291 GB avg-val = 116.33 avg-zval = 31.26 2024/11/23-21:06:55.223534 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280662.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:06:56.088384 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280662.sst): second pass time = 1.49's, 72.479'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 40.885'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3379.733'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.14's, 244.484'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34439.672'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.479'MB/sec zip pipeline throughput = 72.479'MB/sec entries = 929948 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.33 avg-zval = 31.26 usrkeys = 929948 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.33 avg-zval = 31.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929948 bytes = 3487312 } seq expand size = 6509636 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.5122 value = 3.7209 dict = 1.00 all = 3.7899 } Zip/UnZip{ index = 0.0951 value = 0.2687 dict = 1.00 all = 0.2639 } ---------------------------- total value len = 9.762679 GB avg = 0.163 KB (by entry num) total key len = 1.649160 GB avg = 0.028 KB total ukey len = 1.146692 GB avg = 0.016 KB total ukey num = 0.072991123 Billion total entry num = 0.059798852 Billion write speed all = 0.259927851 MB/sec (with seq num) write speed all = 0.249031527 MB/sec (without seq num) 2024/11/23-21:06:56.132586 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280662: 929948 keys, 35414932 bytes, temperature: kWarm 2024/11/23-21:06:56.132658 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367216132621, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280662, "file_size": 35414932, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 29073104, "index_size": 1769280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26038544, "raw_average_key_size": 28, "raw_value_size": 108179252, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929948, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367212, "oldest_key_time": 0, "file_creation_time": 1732367212, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280662, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:06:56.168114 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280663.sst) /dev/shm/ToplingTemp/Topling-188733-FcQXci 2024/11/23-21:06:57.297404 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280663.sst): first pass time = 1.13's, 118.857'MB/sec 2024/11/23-21:06:59.099840 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280663.sst): old prealloc_size = 147639500, real_size = 36321472 2024/11/23-21:06:59.100515 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280663.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925681 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0343 GB avg-val = 116.99 avg-zval = 37.07 2024/11/23-21:06:59.100968 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280663.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:06:59.879062 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280663.sst): second pass time = 1.80's, 60.107'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 49.888'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2725.781'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.355'MB/sec write SST data time = 0.13's, 308.423'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32216.669'MB/sec zip my value time = 1.80's, unzip length = 0.108'GB zip my value throughput = 60.107'MB/sec zip pipeline throughput = 99.857'MB/sec entries = 925681 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.99 avg-zval = 37.07 usrkeys = 925681 avg-key = 20.00 avg-zkey = 1.92 avg-val = 120.11 avg-zval = 37.07 TagRS{ kinds = RS_Key0_TagN bytes = 144720 } TagArray{ size = 412558 bytes = 1547104 } seq expand size = 2887906 multi value expand size = 0 cnt WriteAppend = 18 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0343 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4435 value = 3.1558 dict = 1.00 all = 3.4528 } Zip/UnZip{ index = 0.0958 value = 0.3169 dict = 1.00 all = 0.2896 } ---------------------------- total value len = 9.873866 GB avg = 0.163 KB (by entry num) total key len = 1.675079 GB avg = 0.028 KB total ukey len = 1.165205 GB avg = 0.016 KB total ukey num = 0.073916804 Billion total entry num = 0.060724533 Billion write speed all = 0.263028004 MB/sec (with seq num) write speed all = 0.251963961 MB/sec (without seq num) 2024/11/23-21:06:59.938472 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280663: 925681 keys, 38872529 bytes, temperature: kWarm 2024/11/23-21:06:59.938567 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367219938518, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280663, "file_size": 38872529, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012771944, "table_properties": {"data_size": 34317104, "index_size": 1772736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25919068, "raw_average_key_size": 28, "raw_value_size": 108298802, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925681, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367216, "oldest_key_time": 0, "file_creation_time": 1732367216, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280663, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:07:00.027000 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280664.sst) /dev/shm/ToplingTemp/Topling-188733-xLiMui 2024/11/23-21:07:01.207368 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280664.sst): first pass time = 1.18's, 113.714'MB/sec 2024/11/23-21:07:02.864297 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280664.sst): old prealloc_size = 147639500, real_size = 37070736 2024/11/23-21:07:02.864832 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280664.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924494 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0351 GB avg-val = 117.18 avg-zval = 37.93 2024/11/23-21:07:02.865303 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280664.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:07:03.797985 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280664.sst): second pass time = 1.66's, 65.413'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 42.437'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3446.587'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.23's, 168.723'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 27011.190'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.413'MB/sec zip pipeline throughput = 71.976'MB/sec entries = 924494 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.18 avg-zval = 37.93 usrkeys = 924494 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.30 avg-zval = 37.93 TagRS{ kinds = RS_Key0_TagN bytes = 144560 } TagArray{ size = 412048 bytes = 1545200 } seq expand size = 2884336 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0351 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.4456 value = 3.0891 dict = 1.00 all = 3.3876 } Zip/UnZip{ index = 0.0957 value = 0.3237 dict = 1.00 all = 0.2952 } ---------------------------- total value len = 9.985082 GB avg = 0.162 KB (by entry num) total key len = 1.700965 GB avg = 0.028 KB total ukey len = 1.183695 GB avg = 0.016 KB total ukey num = 0.074841298 Billion total entry num = 0.061649027 Billion write speed all = 0.266126761 MB/sec (with seq num) write speed all = 0.254895278 MB/sec (without seq num) 2024/11/23-21:07:03.852439 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280664: 924494 keys, 39620379 bytes, temperature: kWarm 2024/11/23-21:07:03.852551 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367223852502, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280664, "file_size": 39620379, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4257872545, "table_properties": {"data_size": 35069312, "index_size": 1770112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25885832, "raw_average_key_size": 28, "raw_value_size": 108331952, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924494, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367220, "oldest_key_time": 0, "file_creation_time": 1732367220, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280664, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:07:03.906849 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280665.sst) /dev/shm/ToplingTemp/Topling-188733-CzDXGj 2024/11/23-21:07:04.902344 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280665.sst): first pass time = 1.00's, 134.830'MB/sec 2024/11/23-21:07:06.269772 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280665.sst): old prealloc_size = 147639500, real_size = 34155552 2024/11/23-21:07:06.270383 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280665.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929266 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0322 GB avg-val = 116.43 avg-zval = 34.60 2024/11/23-21:07:06.271188 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280665.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:07:07.122324 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280665.sst): second pass time = 1.37's, 79.199'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.743'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3009.655'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 290.336'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33880.879'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.199'MB/sec zip pipeline throughput = 79.199'MB/sec entries = 929266 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.43 avg-zval = 34.60 usrkeys = 929266 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.43 avg-zval = 34.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929266 bytes = 3136288 } seq expand size = 6504862 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.5132 value = 3.3649 dict = 1.00 all = 3.5187 } Zip/UnZip{ index = 0.0951 value = 0.2972 dict = 1.00 all = 0.2842 } ---------------------------- total value len = 10.099785 GB avg = 0.161 KB (by entry num) total key len = 1.726985 GB avg = 0.028 KB total ukey len = 1.202281 GB avg = 0.016 KB total ukey num = 0.075770564 Billion total entry num = 0.062578293 Billion write speed all = 0.269311056 MB/sec (with seq num) write speed all = 0.257911138 MB/sec (without seq num) 2024/11/23-21:07:07.170298 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280665: 929266 keys, 38144402 bytes, temperature: kWarm 2024/11/23-21:07:07.170368 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367227170333, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280665, "file_size": 38144402, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4190623292, "largest_seqno": 4261176277, "table_properties": {"data_size": 32155216, "index_size": 1767808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26019448, "raw_average_key_size": 28, "raw_value_size": 108198388, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929266, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367223, "oldest_key_time": 0, "file_creation_time": 1732367223, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280665, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:07:07.211695 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280666.sst) /dev/shm/ToplingTemp/Topling-188733-PEIlQj 2024/11/23-21:07:08.253348 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280666.sst): first pass time = 1.04's, 128.855'MB/sec 2024/11/23-21:07:09.588910 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280666.sst): old prealloc_size = 147639500, real_size = 34067232 2024/11/23-21:07:09.589508 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280666.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928089 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0321 GB avg-val = 116.62 avg-zval = 34.54 2024/11/23-21:07:09.590962 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280666.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-21:07:10.436904 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280666.sst): second pass time = 1.33's, 81.108'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.013'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3079.878'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 296.494'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34252.197'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.108'MB/sec zip pipeline throughput = 81.108'MB/sec entries = 928089 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.62 avg-zval = 34.54 usrkeys = 928089 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.62 avg-zval = 34.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928089 bytes = 3248320 } seq expand size = 6496623 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0321 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4443 value = 3.3761 dict = 1.00 all = 3.5164 } Zip/UnZip{ index = 0.0957 value = 0.2962 dict = 1.00 all = 0.2844 } ---------------------------- total value len = 10.214513 GB avg = 0.161 KB (by entry num) total key len = 1.752971 GB avg = 0.028 KB total ukey len = 1.220842 GB avg = 0.016 KB total ukey num = 0.076698653 Billion total entry num = 0.063506382 Billion write speed all = 0.272494740 MB/sec (with seq num) write speed all = 0.260926625 MB/sec (without seq num) 2024/11/23-21:07:10.483536 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280666: 928089 keys, 38169092 bytes, temperature: kWarm 2024/11/23-21:07:10.483603 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367230483568, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280666, "file_size": 38169092, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4100102034, "largest_seqno": 4261176570, "table_properties": {"data_size": 32057808, "index_size": 1777216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986492, "raw_average_key_size": 28, "raw_value_size": 108231298, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928089, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367227, "oldest_key_time": 0, "file_creation_time": 1732367227, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280666, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:07:10.520223 139957282305600 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280667.sst) /dev/shm/ToplingTemp/Topling-188733-9ZbPQg 2024/11/23-21:07:10.646593 139957282305600 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280667.sst): first pass time = 0.13's, 244.667'MB/sec 2024/11/23-21:07:10.942024 139957282305600 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280667.sst): old prealloc_size = 147639500, real_size = 8856720 2024/11/23-21:07:10.942221 139957282305600 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280667.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 214904 prefix = 4 raw-key = 0.0034 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0249 GB zip-val = 0.0084 GB avg-val = 115.83 avg-zval = 39.04 2024/11/23-21:07:10.942429 139957282305600 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280667.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.031 GB 2024/11/23-21:07:11.129272 139957282305600 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00/280667.sst): second pass time = 0.29's, 84.466'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 52.127'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2415.116'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.342'MB/sec write SST data time = 0.03's, 324.400'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.258'MB, throughput = 8679.833'MB/sec zip my value time = 0.29's, unzip length = 0.025'GB zip my value throughput = 84.466'MB/sec zip pipeline throughput = 84.466'MB/sec entries = 214904 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.83 avg-zval = 39.04 usrkeys = 214904 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.83 avg-zval = 39.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 214904 bytes = 698448 } seq expand size = 1504328 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0043 GB value = 0.0249 GB dict = 0.26 MB all = 0.0309 GB } __ZipSize{ index = 0.0004 GB value = 0.0084 GB dict = 0.26 MB all = 0.0098 GB } UnZip/Zip{ index = 10.4136 value = 2.9668 dict = 1.00 all = 3.1665 } Zip/UnZip{ index = 0.0960 value = 0.3371 dict = 1.00 all = 0.3158 } ---------------------------- total value len = 10.240910 GB avg = 0.161 KB (by entry num) total key len = 1.758988 GB avg = 0.028 KB total ukey len = 1.225140 GB avg = 0.016 KB total ukey num = 0.076913557 Billion total entry num = 0.063721286 Billion write speed all = 0.273228475 MB/sec (with seq num) write speed all = 0.261621396 MB/sec (without seq num) 2024/11/23-21:07:11.146614 139957282305600 [compaction_job.cc:1948] [default] [JOB 35786] Generated table #280667: 214904 keys, 9761243 bytes, temperature: kWarm 2024/11/23-21:07:11.146678 139957282305600 EVENT_LOG_v1 {"time_micros": 1732367231146644, "cf_name": "default", "job": 35786, "event": "table_file_creation", "file_number": 280667, "file_size": 9761243, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4220741816, "largest_seqno": 4261174258, "table_properties": {"data_size": 8390080, "index_size": 412736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6017312, "raw_average_key_size": 28, "raw_value_size": 24891856, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 214904, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732367230, "oldest_key_time": 0, "file_creation_time": 1732367230, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 280667, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-21:07:11.160102 139957282305600 [compaction_job.cc:703] job-35786: subcompact[0], size: 0.465871 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-35786/att-00 [280655,280656,280657,280658,280659,280660,280661,280662,280663,280664,280665,280666,280667] 2024/11/23-21:07:11.245348 139957282305600 (Original Log Time 2024/11/23-21:06:24.547618) EVENT_LOG_v1 {"time_micros": 1732367184547589, "job": 35786, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [280587], "files_L3": [280538, 280539, 280540, 280541, 280542, 280543, 280544, 280545, 280546, 280547, 280548, 280601], "score": 0.995451, "input_data_size": 484709709, "oldest_snapshot_seqno": 4262388954} 2024/11/23-21:07:11.245352 139957282305600 (Original Log Time 2024/11/23-21:07:11.240791) [compaction_job.cc:2022] [default] [JOB 35786] Compacted 1@2 + 12@3 files to L3 => 465871035 bytes 2024/11/23-21:07:11.245353 139957282305600 (Original Log Time 2024/11/23-21:07:11.245285) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 10.4 rd, 10.0 wr, level 3, files in(1, 12) out(13 +0 blob) MB in(27.1, 435.2 +0.0 blob) out(444.3 +0.0 blob), read-write-amplify(33.4) write-amplify(16.4) OK, records in: 11381395, records dropped: 18273 output_compression: Snappy 2024/11/23-21:07:11.245359 139957282305600 (Original Log Time 2024/11/23-21:07:11.245328) EVENT_LOG_v1 {"time_micros": 1732367231245308, "cf": "default", "job": 35786, "event": "compaction_finished", "compaction_time_micros": 46612493, "compaction_time_cpu_micros": 19221937, "output_level": 3, "num_output_files": 13, "total_output_size": 465871035, "num_input_records": 11381395, "num_output_records": 11363122, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 371704430, "file_range_sync_nanos": 0, "file_fsync_nanos": 9901, "file_prepare_write_nanos": 41325, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/23-21:07:11.371966 139957282305600 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732367184,"executesMs":46841,"compactionJobId":35786,"attempt":0,"compactionInputRawBytes":1644117594,"compactionInputZipBytes":484709709,"compactionOutputRawBytes":0,"compactionOutputZipBytes":465871035,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-3823cd7bcb4f11ad82d4b6aeb78c3c4a-ec893f106768b477-00"}}, response = 2024/11/23-21:07:11.372067 139957282305600 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-35786/att-00: olev 3, work 46.842 s, result 7.715 ms, install 4.585 ms, input{raw 1.531 GiB zip 462.255 MiB}