2024/11/22-15:46:32.782183 140025188247104 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00: fork to child time = 0.025803 sec 2024/11/22-15:46:32.783259 140025188247104 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00 2024/11/22-15:46:32.791637 140025188247104 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00 2024/11/22-15:46:32.793233 140025188247104 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-15:46:32.793587 140025188247104 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-15:46:32.842398 140025188247104 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-03416/att-00/MANIFEST-206347 2024/11/22-15:46:32.867770 140025188247104 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-03416/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 212342, last_sequence is 3348006553, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 212204 2024/11/22-15:46:32.867802 140025188247104 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-15:46:32.868180 140025188247104 [version_set.cc:5721] Creating manifest 212342 2024/11/22-15:46:32.897835 140025188247104 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-15:46:32.919019 140025188247104 [compaction_job.cc:2437] [default] [JOB 3416] Compacting 1@2 + 12@3 files to L3, score 0.86, subcompactions 1 : 1 2024/11/22-15:46:32.919040 140025188247104 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 2725 Base level 2, inputs: [212329(27MB)], [212282(37MB) 212283(37MB) 212284(37MB) 212285(37MB) 212286(38MB) 212287(38MB) 212288(37MB) 212289(35MB) 212290(35MB) 212291(37MB) 212292(37MB) 212299(37MB)] 2024/11/22-15:46:32.926856 140025188247104 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-15:46:32.926894 140025188247104 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-15:46:32.926921 140025188247104 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-15:46:32.926928 140025188247104 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-15:46:32.926943 140025188247104 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-15:46:33.431528 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212343.sst) /dev/shm/ToplingTemp/Topling-182994-ytfREm 2024/11/22-15:46:34.221503 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212343.sst): first pass time = 0.79's, 169.911'MB/sec 2024/11/22-15:46:38.059058 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212343.sst): old prealloc_size = 147639500, real_size = 35056400 2024/11/22-15:46:38.059649 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212343.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922123 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0330 GB avg-val = 117.55 avg-zval = 35.84 2024/11/22-15:46:38.060440 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212343.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:46:39.183341 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212343.sst): second pass time = 3.84's, 28.256'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 34.714'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3121.318'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.279'MB/sec write SST data time = 0.30's, 131.192'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30688.764'MB/sec zip my value time = 3.84's, unzip length = 0.108'GB zip my value throughput = 28.256'MB/sec zip pipeline throughput = 121.433'MB/sec entries = 922123 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.55 avg-zval = 35.84 usrkeys = 922123 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.55 avg-zval = 35.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922123 bytes = 3112176 } seq expand size = 6454861 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0390 GB } UnZip/Zip{ index = 10.3764 value = 3.2800 dict = 1.00 all = 3.4391 } Zip/UnZip{ index = 0.0964 value = 0.3049 dict = 1.00 all = 0.2908 } ---------------------------- total value len = 0.114853 GB avg = 0.125 KB (by entry num) total key len = 0.025819 GB avg = 0.028 KB total ukey len = 0.018442 GB avg = 0.020 KB total ukey num = 0.000922123 Billion total entry num = 0.000922123 Billion write speed all = 0.005659485 MB/sec (with seq num) write speed all = 0.005362698 MB/sec (without seq num) 2024/11/22-15:46:39.222466 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212343: 922123 keys, 39027076 bytes, temperature: kWarm 2024/11/22-15:46:39.222584 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261599222509, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212343, "file_size": 39027076, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702550, "largest_seqno": 3347952194, "table_properties": {"data_size": 33048320, "index_size": 1777344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25819444, "raw_average_key_size": 28, "raw_value_size": 108398358, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922123, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261593, "oldest_key_time": 0, "file_creation_time": 1732261593, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212343, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:46:39.295230 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212344.sst) /dev/shm/ToplingTemp/Topling-182994-jBvB0j 2024/11/22-15:46:40.394303 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212344.sst): first pass time = 1.10's, 122.123'MB/sec 2024/11/22-15:46:42.140654 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212344.sst): old prealloc_size = 147639500, real_size = 35179872 2024/11/22-15:46:42.141185 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212344.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922120 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.55 avg-zval = 36.00 2024/11/22-15:46:42.141943 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212344.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:46:43.829991 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212344.sst): second pass time = 1.74's, 62.124'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.69's, 23.176'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3459.395'MB/sec (index lex order gen) rebuild zvType time = 1.44's, 0.160'MB/sec write SST data time = 0.25's, 156.396'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30097.900'MB/sec zip my value time = 1.74's, unzip length = 0.108'GB zip my value throughput = 62.124'MB/sec zip pipeline throughput = 62.124'MB/sec entries = 922120 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.55 avg-zval = 36.00 usrkeys = 922120 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.55 avg-zval = 36.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922120 bytes = 3112176 } seq expand size = 6454840 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0332 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.5127 value = 3.2655 dict = 1.00 all = 3.4282 } Zip/UnZip{ index = 0.0951 value = 0.3062 dict = 1.00 all = 0.2917 } ---------------------------- total value len = 0.229706 GB avg = 0.125 KB (by entry num) total key len = 0.051639 GB avg = 0.028 KB total ukey len = 0.036885 GB avg = 0.020 KB total ukey num = 0.001844243 Billion total entry num = 0.001844243 Billion write speed all = 0.011316852 MB/sec (with seq num) write speed all = 0.010723388 MB/sec (without seq num) 2024/11/22-15:46:43.867384 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212344: 922120 keys, 39150684 bytes, temperature: kWarm 2024/11/22-15:46:43.867465 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261603867421, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212344, "file_size": 39150684, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702575, "largest_seqno": 3347952518, "table_properties": {"data_size": 33194832, "index_size": 1754304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25819360, "raw_average_key_size": 28, "raw_value_size": 108398376, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922120, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261599, "oldest_key_time": 0, "file_creation_time": 1732261599, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212344, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:46:43.938214 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212345.sst) /dev/shm/ToplingTemp/Topling-182994-sjb2gn 2024/11/22-15:46:45.318245 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212345.sst): first pass time = 1.38's, 97.261'MB/sec 2024/11/22-15:46:47.217674 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212345.sst): old prealloc_size = 147639500, real_size = 34908144 2024/11/22-15:46:47.218216 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212345.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922954 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0329 GB avg-val = 117.42 avg-zval = 35.66 2024/11/22-15:46:47.218985 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212345.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:46:48.409751 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212345.sst): second pass time = 1.90's, 57.102'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.19's, 32.617'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3400.996'MB/sec (index lex order gen) rebuild zvType time = 0.96's, 0.241'MB/sec write SST data time = 0.23's, 166.779'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 28697.837'MB/sec zip my value time = 1.90's, unzip length = 0.108'GB zip my value throughput = 57.102'MB/sec zip pipeline throughput = 67.923'MB/sec entries = 922954 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.42 avg-zval = 35.66 usrkeys = 922954 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.42 avg-zval = 35.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922954 bytes = 3114992 } seq expand size = 6460678 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0329 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4433 value = 3.2931 dict = 1.00 all = 3.4520 } Zip/UnZip{ index = 0.0958 value = 0.3037 dict = 1.00 all = 0.2897 } ---------------------------- total value len = 0.344542 GB avg = 0.125 KB (by entry num) total key len = 0.077482 GB avg = 0.028 KB total ukey len = 0.055344 GB avg = 0.020 KB total ukey num = 0.002767197 Billion total entry num = 0.002767197 Billion write speed all = 0.016972387 MB/sec (with seq num) write speed all = 0.016082087 MB/sec (without seq num) 2024/11/22-15:46:48.448716 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212345: 922954 keys, 38880936 bytes, temperature: kWarm 2024/11/22-15:46:48.448807 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261608448754, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212345, "file_size": 38880936, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702604, "largest_seqno": 3347952281, "table_properties": {"data_size": 32909664, "index_size": 1767552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842712, "raw_average_key_size": 28, "raw_value_size": 108375096, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922954, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261603, "oldest_key_time": 0, "file_creation_time": 1732261603, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212345, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:46:48.517405 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212346.sst) /dev/shm/ToplingTemp/Topling-182994-5Lpxin 2024/11/22-15:46:49.778810 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212346.sst): first pass time = 1.26's, 106.407'MB/sec 2024/11/22-15:46:51.800438 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212346.sst): old prealloc_size = 147639500, real_size = 34945184 2024/11/22-15:46:51.801002 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212346.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922445 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0330 GB avg-val = 117.50 avg-zval = 35.73 2024/11/22-15:46:51.801818 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212346.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:46:52.903458 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212346.sst): second pass time = 2.02's, 53.631'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 35.283'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.370'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.265'MB/sec write SST data time = 0.23's, 167.928'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30810.536'MB/sec zip my value time = 2.02's, unzip length = 0.108'GB zip my value throughput = 53.631'MB/sec zip pipeline throughput = 118.762'MB/sec entries = 922445 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.50 avg-zval = 35.73 usrkeys = 922445 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.50 avg-zval = 35.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922445 bytes = 3113264 } seq expand size = 6457115 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.5114 value = 3.2886 dict = 1.00 all = 3.4488 } Zip/UnZip{ index = 0.0951 value = 0.3041 dict = 1.00 all = 0.2900 } ---------------------------- total value len = 0.459389 GB avg = 0.125 KB (by entry num) total key len = 0.103310 GB avg = 0.028 KB total ukey len = 0.073793 GB avg = 0.020 KB total ukey num = 0.003689642 Billion total entry num = 0.003689642 Billion write speed all = 0.022625772 MB/sec (with seq num) write speed all = 0.021438906 MB/sec (without seq num) 2024/11/22-15:46:52.940219 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212346: 922445 keys, 38916822 bytes, temperature: kWarm 2024/11/22-15:46:52.940302 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261612940258, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212346, "file_size": 38916822, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273702634, "largest_seqno": 3347952460, "table_properties": {"data_size": 32959248, "index_size": 1755136, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25828460, "raw_average_key_size": 28, "raw_value_size": 108389290, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922445, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261608, "oldest_key_time": 0, "file_creation_time": 1732261608, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212346, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:46:52.987663 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212347.sst) /dev/shm/ToplingTemp/Topling-182994-h7Ayin 2024/11/22-15:46:54.020531 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212347.sst): first pass time = 1.03's, 129.952'MB/sec 2024/11/22-15:46:55.594237 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212347.sst): old prealloc_size = 147639500, real_size = 36563088 2024/11/22-15:46:55.594804 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212347.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922326 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0346 GB avg-val = 117.52 avg-zval = 37.49 2024/11/22-15:46:55.595557 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212347.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:46:56.624488 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212347.sst): second pass time = 1.57's, 68.937'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 39.349'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3210.498'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.256'MB/sec write SST data time = 0.13's, 310.626'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 36085.468'MB/sec zip my value time = 1.57's, unzip length = 0.108'GB zip my value throughput = 68.937'MB/sec zip pipeline throughput = 68.937'MB/sec entries = 922326 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 37.49 usrkeys = 922326 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.52 avg-zval = 37.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922326 bytes = 3112864 } seq expand size = 6456282 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.5123 value = 3.1348 dict = 1.00 all = 3.3109 } Zip/UnZip{ index = 0.0951 value = 0.3190 dict = 1.00 all = 0.3020 } ---------------------------- total value len = 0.574238 GB avg = 0.125 KB (by entry num) total key len = 0.129135 GB avg = 0.028 KB total ukey len = 0.092239 GB avg = 0.020 KB total ukey num = 0.004611968 Billion total entry num = 0.004611968 Billion write speed all = 0.028277963 MB/sec (with seq num) write speed all = 0.026794630 MB/sec (without seq num) 2024/11/22-15:46:56.661460 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212347: 922326 keys, 40538258 bytes, temperature: kWarm 2024/11/22-15:46:56.661525 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261616661491, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212347, "file_size": 40538258, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868609, "largest_seqno": 3347952330, "table_properties": {"data_size": 34577600, "index_size": 1754752, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25825128, "raw_average_key_size": 28, "raw_value_size": 108392724, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922326, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261612, "oldest_key_time": 0, "file_creation_time": 1732261612, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212347, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:46:56.698871 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212348.sst) /dev/shm/ToplingTemp/Topling-182994-nTuYjl 2024/11/22-15:46:57.630851 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212348.sst): first pass time = 0.93's, 144.020'MB/sec 2024/11/22-15:46:59.256305 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212348.sst): old prealloc_size = 147639500, real_size = 36553952 2024/11/22-15:46:59.256854 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212348.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922962 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0346 GB avg-val = 117.42 avg-zval = 37.44 2024/11/22-15:46:59.257590 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212348.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:47:00.105295 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212348.sst): second pass time = 1.62's, 66.719'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 47.660'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3339.356'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 305.874'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 38225.547'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.719'MB/sec zip pipeline throughput = 66.719'MB/sec entries = 922962 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.42 avg-zval = 37.44 usrkeys = 922962 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.20 avg-zval = 37.44 TagRS{ kinds = RS_Key0_TagN bytes = 144320 } TagArray{ size = 893924 bytes = 2905264 } seq expand size = 6257468 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0405 GB } UnZip/Zip{ index = 10.4464 value = 3.1362 dict = 1.00 all = 3.3171 } Zip/UnZip{ index = 0.0957 value = 0.3189 dict = 1.00 all = 0.3015 } ---------------------------- total value len = 0.688870 GB avg = 0.124 KB (by entry num) total key len = 0.154978 GB avg = 0.028 KB total ukey len = 0.110699 GB avg = 0.020 KB total ukey num = 0.005534930 Billion total entry num = 0.005534930 Billion write speed all = 0.033920795 MB/sec (with seq num) write speed all = 0.032140861 MB/sec (without seq num) 2024/11/22-15:47:00.141507 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212348: 922962 keys, 40461952 bytes, temperature: kWarm 2024/11/22-15:47:00.141568 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261620141537, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212348, "file_size": 40461952, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273699977, "table_properties": {"data_size": 34555984, "index_size": 1767040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25842936, "raw_average_key_size": 28, "raw_value_size": 108374796, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922962, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261616, "oldest_key_time": 0, "file_creation_time": 1732261616, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212348, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:47:00.177830 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212349.sst) /dev/shm/ToplingTemp/Topling-182994-88wvyl 2024/11/22-15:47:01.014275 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212349.sst): first pass time = 0.84's, 160.470'MB/sec 2024/11/22-15:47:02.508333 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212349.sst): old prealloc_size = 147639500, real_size = 36465920 2024/11/22-15:47:02.508883 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212349.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923701 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1084 GB zip-val = 0.0345 GB avg-val = 117.30 avg-zval = 37.34 2024/11/22-15:47:02.509473 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212349.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:47:03.299065 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212349.sst): second pass time = 1.49's, 72.556'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.79's, 50.133'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3372.149'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.349'MB/sec write SST data time = 0.13's, 307.520'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32171.856'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.556'MB/sec zip pipeline throughput = 72.556'MB/sec entries = 923701 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.30 avg-zval = 37.34 usrkeys = 923701 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.36 avg-zval = 37.34 TagRS{ kinds = RS_Key0_TagN bytes = 144440 } TagArray{ size = 667421 bytes = 2169136 } seq expand size = 4671947 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0345 GB dict = 1.08 MB all = 0.0396 GB } UnZip/Zip{ index = 10.5820 value = 3.1417 dict = 1.00 all = 3.3858 } Zip/UnZip{ index = 0.0945 value = 0.3183 dict = 1.00 all = 0.2953 } ---------------------------- total value len = 0.801896 GB avg = 0.124 KB (by entry num) total key len = 0.180842 GB avg = 0.028 KB total ukey len = 0.129173 GB avg = 0.020 KB total ukey num = 0.006458631 Billion total entry num = 0.006458631 Billion write speed all = 0.039498781 MB/sec (with seq num) write speed all = 0.037422068 MB/sec (without seq num) 2024/11/22-15:47:03.334194 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212349: 923701 keys, 39640872 bytes, temperature: kWarm 2024/11/22-15:47:03.334255 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261623334223, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212349, "file_size": 39640872, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3273700833, "table_properties": {"data_size": 34489008, "index_size": 1745792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25863628, "raw_average_key_size": 28, "raw_value_size": 108354186, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923701, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261620, "oldest_key_time": 0, "file_creation_time": 1732261620, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212349, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:47:03.371968 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212350.sst) /dev/shm/ToplingTemp/Topling-182994-8D1lLn 2024/11/22-15:47:04.320539 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212350.sst): first pass time = 0.95's, 141.499'MB/sec 2024/11/22-15:47:05.884040 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212350.sst): old prealloc_size = 147639500, real_size = 36499792 2024/11/22-15:47:05.884574 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212350.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927146 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.76 avg-zval = 37.23 2024/11/22-15:47:05.884613 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212350.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:47:06.523515 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212350.sst): second pass time = 1.56's, 69.271'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.422'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.376'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.449'MB/sec write SST data time = 0.12's, 303.086'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34831.367'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.271'MB/sec zip pipeline throughput = 69.271'MB/sec entries = 927146 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.76 avg-zval = 37.23 usrkeys = 927146 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.76 avg-zval = 37.23 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5827 value = 3.1365 dict = 1.00 all = 3.5927 } Zip/UnZip{ index = 0.0945 value = 0.3188 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 0.910154 GB avg = 0.123 KB (by entry num) total key len = 0.206802 GB avg = 0.028 KB total ukey len = 0.147716 GB avg = 0.020 KB total ukey num = 0.007385777 Billion total entry num = 0.007385777 Billion write speed all = 0.044887524 MB/sec (with seq num) write speed all = 0.042513003 MB/sec (without seq num) 2024/11/22-15:47:06.556833 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212350: 927146 keys, 37358184 bytes, temperature: kWarm 2024/11/22-15:47:06.556895 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261626556863, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212350, "file_size": 37358184, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34515648, "index_size": 1752192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960088, "raw_average_key_size": 28, "raw_value_size": 108257680, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927146, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261623, "oldest_key_time": 0, "file_creation_time": 1732261623, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212350, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:47:06.592127 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212351.sst) /dev/shm/ToplingTemp/Topling-182994-cI60qn 2024/11/22-15:47:07.361888 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212351.sst): first pass time = 0.77's, 174.370'MB/sec 2024/11/22-15:47:08.910036 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212351.sst): old prealloc_size = 147639500, real_size = 36383024 2024/11/22-15:47:08.910580 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212351.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926830 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.81 avg-zval = 37.12 2024/11/22-15:47:08.910623 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212351.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:47:09.547962 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212351.sst): second pass time = 1.55's, 69.962'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.381'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3358.501'MB/sec (index lex order gen) rebuild zvType time = 0.51's, 0.450'MB/sec write SST data time = 0.12's, 304.238'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35405.735'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.962'MB/sec zip pipeline throughput = 69.962'MB/sec entries = 926830 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.81 avg-zval = 37.12 usrkeys = 926830 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.81 avg-zval = 37.12 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5822 value = 3.1473 dict = 1.00 all = 3.6040 } Zip/UnZip{ index = 0.0945 value = 0.3177 dict = 1.00 all = 0.2775 } ---------------------------- total value len = 1.018420 GB avg = 0.123 KB (by entry num) total key len = 0.232753 GB avg = 0.028 KB total ukey len = 0.166252 GB avg = 0.020 KB total ukey num = 0.008312607 Billion total entry num = 0.008312607 Billion write speed all = 0.050275277 MB/sec (with seq num) write speed all = 0.047603106 MB/sec (without seq num) 2024/11/22-15:47:09.584460 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212351: 926830 keys, 37241240 bytes, temperature: kWarm 2024/11/22-15:47:09.584527 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261629584492, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212351, "file_size": 37241240, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34399456, "index_size": 1751680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951240, "raw_average_key_size": 28, "raw_value_size": 108266564, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926830, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261626, "oldest_key_time": 0, "file_creation_time": 1732261626, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212351, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:47:09.618385 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212352.sst) /dev/shm/ToplingTemp/Topling-182994-UU6XBn 2024/11/22-15:47:10.485845 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212352.sst): first pass time = 0.87's, 154.732'MB/sec 2024/11/22-15:47:11.817479 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212352.sst): old prealloc_size = 147639500, real_size = 35126112 2024/11/22-15:47:11.818029 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212352.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922199 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1084 GB zip-val = 0.0332 GB avg-val = 117.54 avg-zval = 35.95 2024/11/22-15:47:11.818744 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212352.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:47:12.687898 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212352.sst): second pass time = 1.33's, 81.445'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 44.585'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3300.668'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.311'MB/sec write SST data time = 0.13's, 301.366'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32411.775'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.445'MB/sec zip pipeline throughput = 81.445'MB/sec entries = 922199 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.54 avg-zval = 35.95 usrkeys = 922199 avg-key = 20.00 avg-zkey = 1.89 avg-val = 123.55 avg-zval = 35.95 TagRS{ kinds = RS_Key0_TagN bytes = 144200 } TagArray{ size = 792249 bytes = 2673856 } seq expand size = 5545743 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0332 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5815 value = 3.2696 dict = 1.00 all = 3.4586 } Zip/UnZip{ index = 0.0945 value = 0.3058 dict = 1.00 all = 0.2891 } ---------------------------- total value len = 1.132362 GB avg = 0.123 KB (by entry num) total key len = 0.258575 GB avg = 0.028 KB total ukey len = 0.184696 GB avg = 0.020 KB total ukey num = 0.009234806 Billion total entry num = 0.009234806 Billion write speed all = 0.055884274 MB/sec (with seq num) write speed all = 0.052916028 MB/sec (without seq num) 2024/11/22-15:47:12.721891 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212352: 922199 keys, 38806484 bytes, temperature: kWarm 2024/11/22-15:47:12.721953 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261632721920, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212352, "file_size": 38806484, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3347952550, "table_properties": {"data_size": 33152336, "index_size": 1743040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25821572, "raw_average_key_size": 28, "raw_value_size": 108396210, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922199, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261629, "oldest_key_time": 0, "file_creation_time": 1732261629, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212352, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:47:12.755721 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212353.sst) /dev/shm/ToplingTemp/Topling-182994-lEUNlj 2024/11/22-15:47:13.686859 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212353.sst): first pass time = 0.93's, 144.149'MB/sec 2024/11/22-15:47:15.027696 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212353.sst): old prealloc_size = 147639500, real_size = 34829040 2024/11/22-15:47:15.028246 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212353.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923138 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0328 GB avg-val = 117.39 avg-zval = 35.58 2024/11/22-15:47:15.028999 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212353.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:47:15.926297 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212353.sst): second pass time = 1.34's, 80.894'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 43.182'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3326.443'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.299'MB/sec write SST data time = 0.13's, 305.689'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 29975.970'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.894'MB/sec zip pipeline throughput = 80.894'MB/sec entries = 923138 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 35.58 usrkeys = 923138 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 35.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923138 bytes = 3115600 } seq expand size = 6461966 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0328 GB dict = 1.08 MB all = 0.0388 GB } UnZip/Zip{ index = 10.5105 value = 3.2998 dict = 1.00 all = 3.4590 } Zip/UnZip{ index = 0.0951 value = 0.3030 dict = 1.00 all = 0.2891 } ---------------------------- total value len = 1.247194 GB avg = 0.123 KB (by entry num) total key len = 0.284422 GB avg = 0.028 KB total ukey len = 0.203159 GB avg = 0.020 KB total ukey num = 0.010157944 Billion total entry num = 0.010157944 Billion write speed all = 0.061528422 MB/sec (with seq num) write speed all = 0.058263885 MB/sec (without seq num) 2024/11/22-15:47:15.960014 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212353: 923138 keys, 38802458 bytes, temperature: kWarm 2024/11/22-15:47:15.960071 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261635960039, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212353, "file_size": 38802458, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701373, "largest_seqno": 3347952676, "table_properties": {"data_size": 32841440, "index_size": 1756608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847864, "raw_average_key_size": 28, "raw_value_size": 108369944, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923138, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261632, "oldest_key_time": 0, "file_creation_time": 1732261632, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212353, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:47:15.994839 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212354.sst) /dev/shm/ToplingTemp/Topling-182994-hsWKun 2024/11/22-15:47:16.791341 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212354.sst): first pass time = 0.80's, 168.516'MB/sec 2024/11/22-15:47:18.127888 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212354.sst): old prealloc_size = 147639500, real_size = 35092592 2024/11/22-15:47:18.128424 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212354.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922015 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0331 GB avg-val = 117.57 avg-zval = 35.91 2024/11/22-15:47:18.129196 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212354.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-15:47:19.018591 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212354.sst): second pass time = 1.34's, 81.144'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 43.858'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3391.344'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.301'MB/sec write SST data time = 0.13's, 311.975'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35204.315'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.144'MB/sec zip pipeline throughput = 81.144'MB/sec entries = 922015 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.57 avg-zval = 35.91 usrkeys = 922015 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.57 avg-zval = 35.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922015 bytes = 3111808 } seq expand size = 6454105 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5122 value = 3.2742 dict = 1.00 all = 3.4359 } Zip/UnZip{ index = 0.0951 value = 0.3054 dict = 1.00 all = 0.2910 } ---------------------------- total value len = 1.362050 GB avg = 0.123 KB (by entry num) total key len = 0.310239 GB avg = 0.028 KB total ukey len = 0.221599 GB avg = 0.020 KB total ukey num = 0.011079959 Billion total entry num = 0.011079959 Billion write speed all = 0.067171180 MB/sec (with seq num) write speed all = 0.063610771 MB/sec (without seq num) 2024/11/22-15:47:19.053713 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212354: 922015 keys, 39063240 bytes, temperature: kWarm 2024/11/22-15:47:19.053787 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261639053743, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212354, "file_size": 39063240, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701413, "largest_seqno": 3347951748, "table_properties": {"data_size": 33107744, "index_size": 1754176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25816420, "raw_average_key_size": 28, "raw_value_size": 108401422, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922015, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261635, "oldest_key_time": 0, "file_creation_time": 1732261635, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212354, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:47:19.089849 140025188247104 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212355.sst) /dev/shm/ToplingTemp/Topling-182994-aItnRj 2024/11/22-15:47:19.163995 140025188247104 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212355.sst): first pass time = 0.07's, 237.820'MB/sec 2024/11/22-15:47:19.323961 140025188247104 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212355.sst): old prealloc_size = 147639500, real_size = 5261712 2024/11/22-15:47:19.324076 140025188247104 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212355.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 120725 prefix = 4 raw-key = 0.0019 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0142 GB zip-val = 0.0050 GB avg-val = 118.00 avg-zval = 41.41 2024/11/22-15:47:19.324216 140025188247104 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212355.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.018 GB 2024/11/22-15:47:19.436329 140025188247104 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00/212355.sst): second pass time = 0.16's, 89.337'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.11's, 51.259'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2490.977'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.316'MB/sec write SST data time = 0.02's, 342.139'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.149'MB, throughput = 4729.737'MB/sec zip my value time = 0.16's, unzip length = 0.014'GB zip my value throughput = 89.337'MB/sec zip pipeline throughput = 89.337'MB/sec entries = 120725 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 41.41 usrkeys = 120725 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 41.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 120725 bytes = 377280 } seq expand size = 845075 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0024 GB value = 0.0142 GB dict = 0.15 MB all = 0.0176 GB } __ZipSize{ index = 0.0002 GB value = 0.0050 GB dict = 0.15 MB all = 0.0058 GB } UnZip/Zip{ index = 10.3873 value = 2.8497 dict = 1.00 all = 3.0604 } Zip/UnZip{ index = 0.0963 value = 0.3509 dict = 1.00 all = 0.3268 } ---------------------------- total value len = 1.377140 GB avg = 0.123 KB (by entry num) total key len = 0.313619 GB avg = 0.028 KB total ukey len = 0.224014 GB avg = 0.020 KB total ukey num = 0.011200684 Billion total entry num = 0.011200684 Billion write speed all = 0.067911967 MB/sec (with seq num) write speed all = 0.064312824 MB/sec (without seq num) 2024/11/22-15:47:19.446898 140025188247104 [compaction_job.cc:1948] [default] [JOB 3416] Generated table #212355: 120725 keys, 5759305 bytes, temperature: kWarm 2024/11/22-15:47:19.446950 140025188247104 EVENT_LOG_v1 {"time_micros": 1732261639446921, "cf_name": "default", "job": 3416, "event": "table_file_creation", "file_number": 212355, "file_size": 5759305, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3306460366, "largest_seqno": 3331028843, "table_properties": {"data_size": 4998912, "index_size": 232448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3380300, "raw_average_key_size": 28, "raw_value_size": 14245550, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 120725, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732261639, "oldest_key_time": 0, "file_creation_time": 1732261639, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 212355, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-15:47:19.459515 140025188247104 [compaction_job.cc:703] job-03416: subcompact[0], size: 0.473648 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-03416/att-00 [212343,212344,212345,212346,212347,212348,212349,212350,212351,212352,212353,212354,212355] 2024/11/22-15:47:19.542812 140025188247104 (Original Log Time 2024/11/22-15:46:32.919115) EVENT_LOG_v1 {"time_micros": 1732261592919059, "job": 3416, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [212329], "files_L3": [212282, 212283, 212284, 212285, 212286, 212287, 212288, 212289, 212290, 212291, 212292, 212299], "score": 0.863191, "input_data_size": 497436470, "oldest_snapshot_seqno": 3348006527} 2024/11/22-15:47:19.542815 140025188247104 (Original Log Time 2024/11/22-15:47:19.542135) [compaction_job.cc:2022] [default] [JOB 3416] Compacted 1@2 + 12@3 files to L3 => 473647511 bytes 2024/11/22-15:47:19.542817 140025188247104 (Original Log Time 2024/11/22-15:47:19.542734) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 10.7 rd, 10.2 wr, level 3, files in(1, 12) out(13 +0 blob) MB in(27.2, 447.2 +0.0 blob) out(451.7 +0.0 blob), read-write-amplify(34.1) write-amplify(16.6) OK, records in: 11313144, records dropped: 112460 output_compression: Snappy 2024/11/22-15:47:19.542822 140025188247104 (Original Log Time 2024/11/22-15:47:19.542776) EVENT_LOG_v1 {"time_micros": 1732261639542757, "cf": "default", "job": 3416, "event": "compaction_finished", "compaction_time_micros": 46540404, "compaction_time_cpu_micros": 19485114, "output_level": 3, "num_output_files": 13, "total_output_size": 473647511, "num_input_records": 11313144, "num_output_records": 11200684, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 335859278, "file_range_sync_nanos": 0, "file_fsync_nanos": 12901, "file_prepare_write_nanos": 28958, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/22-15:47:19.673797 140025188247104 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732261592,"executesMs":46754,"compactionJobId":3416,"attempt":0,"compactionInputRawBytes":1644208668,"compactionInputZipBytes":497436470,"compactionOutputRawBytes":0,"compactionOutputZipBytes":473647511,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-d16fa7496a8140b8bfec55bd055dcf2e-bb5f861ad484bb41-00"}}, response = 2024/11/22-15:47:19.673867 140025188247104 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-03416/att-00: olev 3, work 46.755 s, result 4.818 ms, install 0.705 ms, input{raw 1.531 GiB zip 474.392 MiB}