2024/11/24-00:22:53.507963 139957433374272 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00: fork to child time = 0.016674 sec 2024/11/24-00:22:53.508792 139957433374272 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00 2024/11/24-00:22:53.514559 139957433374272 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00 2024/11/24-00:22:53.515668 139957433374272 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-00:22:53.515926 139957433374272 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-00:22:53.570373 139957433374272 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38509/att-00/MANIFEST-206347 2024/11/24-00:22:53.590595 139957433374272 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38509/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 285668, last_sequence is 4346804510, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 285555 2024/11/24-00:22:53.590603 139957433374272 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-00:22:53.590726 139957433374272 [version_set.cc:5721] Creating manifest 285668 2024/11/24-00:22:53.623935 139957433374272 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-00:22:53.643043 139957433374272 [compaction_job.cc:2437] [default] [JOB 38509] Compacting 2@2 + 13@3 files to L3, score 0.47, subcompactions 1 : 1 2024/11/24-00:22:53.643061 139957433374272 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 28850 Base level 2, inputs: [285666(27MB) 285661(2506KB)], [285623(35MB) 285624(33MB) 285625(31MB) 285626(33MB) 285627(35MB) 285628(35MB) 285629(35MB) 285630(35MB) 285631(34MB) 285632(34MB) 285633(6447KB) 285604(34MB) 285605(29MB)] 2024/11/24-00:22:53.650058 139957433374272 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:22:53.650089 139957433374272 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:22:53.650110 139957433374272 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-00:22:53.650116 139957433374272 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:22:53.650130 139957433374272 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-00:22:54.233694 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285669.sst) /dev/shm/ToplingTemp/Topling-225811-9JbdLX 2024/11/24-00:22:54.816406 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285669.sst): first pass time = 0.58's, 230.350'MB/sec 2024/11/24-00:22:56.810124 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285669.sst): old prealloc_size = 147639500, real_size = 33576480 2024/11/24-00:22:56.810724 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285669.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927168 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.76 avg-zval = 34.05 2024/11/24-00:22:56.811574 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285669.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:22:58.052784 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285669.sst): second pass time = 1.99's, 54.319'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.24's, 30.230'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3072.946'MB/sec (index lex order gen) rebuild zvType time = 1.00's, 0.231'MB/sec write SST data time = 0.24's, 157.093'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32490.726'MB/sec zip my value time = 1.99's, unzip length = 0.108'GB zip my value throughput = 54.319'MB/sec zip pipeline throughput = 107.694'MB/sec entries = 927168 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.76 avg-zval = 34.05 usrkeys = 927168 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.76 avg-zval = 34.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927168 bytes = 3129200 } seq expand size = 6490176 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4452 value = 3.4292 dict = 1.00 all = 3.5730 } Zip/UnZip{ index = 0.0957 value = 0.2916 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 8.963367 GB avg = 0.168 KB (by entry num) total key len = 1.466782 GB avg = 0.028 KB total ukey len = 1.016421 GB avg = 0.015 KB total ukey num = 0.066477603 Billion total entry num = 0.053285332 Billion write speed all = 0.187370732 MB/sec (with seq num) write speed all = 0.179712846 MB/sec (without seq num) 2024/11/24-00:22:58.103073 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285669: 927168 keys, 37564212 bytes, temperature: kWarm 2024/11/24-00:22:58.103509 139957433374272 EVENT_LOG_v1 {"time_micros": 1732378978103282, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285669, "file_size": 37564212, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4346790264, "table_properties": {"data_size": 31569232, "index_size": 1775296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960704, "raw_average_key_size": 28, "raw_value_size": 108257108, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927168, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378974, "oldest_key_time": 0, "file_creation_time": 1732378974, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285669, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:22:58.188541 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285670.sst) /dev/shm/ToplingTemp/Topling-225811-rxL3SY 2024/11/24-00:22:59.524614 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285670.sst): first pass time = 1.34's, 100.461'MB/sec 2024/11/24-00:23:01.064116 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285670.sst): old prealloc_size = 147639500, real_size = 31157152 2024/11/24-00:23:01.064723 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285670.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 938639 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1079 GB zip-val = 0.0291 GB avg-val = 114.99 avg-zval = 31.04 2024/11/24-00:23:01.065482 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285670.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:23:02.647818 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285670.sst): second pass time = 1.54's, 70.167'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.58's, 22.209'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3081.046'MB/sec (index lex order gen) rebuild zvType time = 1.41's, 0.167'MB/sec write SST data time = 0.18's, 197.956'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 39864.771'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.167'MB/sec zip pipeline throughput = 70.167'MB/sec entries = 938639 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.99 avg-zval = 31.04 usrkeys = 938639 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.99 avg-zval = 31.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 938639 bytes = 3167920 } seq expand size = 6570473 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.5112 value = 3.7045 dict = 1.00 all = 3.8159 } Zip/UnZip{ index = 0.0951 value = 0.2699 dict = 1.00 all = 0.2621 } ---------------------------- total value len = 9.077873 GB avg = 0.167 KB (by entry num) total key len = 1.493064 GB avg = 0.028 KB total ukey len = 1.035194 GB avg = 0.015 KB total ukey num = 0.067416242 Billion total entry num = 0.054223971 Billion write speed all = 0.189884226 MB/sec (with seq num) write speed all = 0.182092086 MB/sec (without seq num) 2024/11/24-00:23:02.698233 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285670: 938639 keys, 35172860 bytes, temperature: kWarm 2024/11/24-00:23:02.698311 139957433374272 EVENT_LOG_v1 {"time_micros": 1732378982698264, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285670, "file_size": 35172860, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4346790218, "table_properties": {"data_size": 29136336, "index_size": 1785984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26281892, "raw_average_key_size": 28, "raw_value_size": 107935862, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 938639, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378978, "oldest_key_time": 0, "file_creation_time": 1732378978, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285670, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:02.737200 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285671.sst) /dev/shm/ToplingTemp/Topling-225811-E9x89X 2024/11/24-00:23:04.084240 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285671.sst): first pass time = 1.35's, 99.642'MB/sec 2024/11/24-00:23:05.744327 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285671.sst): old prealloc_size = 147639500, real_size = 29293216 2024/11/24-00:23:05.745031 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285671.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936726 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0273 GB avg-val = 115.28 avg-zval = 29.11 2024/11/24-00:23:05.745831 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285671.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:23:06.749784 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285671.sst): second pass time = 1.66's, 65.088'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 33.122'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2613.884'MB/sec (index lex order gen) rebuild zvType time = 0.84's, 0.277'MB/sec write SST data time = 0.16's, 207.397'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 37797.010'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.088'MB/sec zip pipeline throughput = 65.088'MB/sec entries = 936726 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.28 avg-zval = 29.11 usrkeys = 936726 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.28 avg-zval = 29.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936726 bytes = 3161472 } seq expand size = 6557082 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0273 GB dict = 1.08 MB all = 0.0333 GB } UnZip/Zip{ index = 10.4437 value = 3.9607 dict = 1.00 all = 4.0304 } Zip/UnZip{ index = 0.0958 value = 0.2525 dict = 1.00 all = 0.2481 } ---------------------------- total value len = 9.192420 GB avg = 0.167 KB (by entry num) total key len = 1.519292 GB avg = 0.028 KB total ukey len = 1.053929 GB avg = 0.015 KB total ukey num = 0.068352968 Billion total entry num = 0.055160697 Billion write speed all = 0.192398767 MB/sec (with seq num) write speed all = 0.184472601 MB/sec (without seq num) 2024/11/24-00:23:06.796119 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285671: 936726 keys, 33301452 bytes, temperature: kWarm 2024/11/24-00:23:06.796191 139957433374272 EVENT_LOG_v1 {"time_micros": 1732378986796155, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285671, "file_size": 33301452, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679393, "largest_seqno": 4346789619, "table_properties": {"data_size": 27264976, "index_size": 1793856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26228328, "raw_average_key_size": 28, "raw_value_size": 107989408, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936726, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378982, "oldest_key_time": 0, "file_creation_time": 1732378982, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285671, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:06.835182 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285672.sst) /dev/shm/ToplingTemp/Topling-225811-gN59NZ 2024/11/24-00:23:07.843342 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285672.sst): first pass time = 1.01's, 133.137'MB/sec 2024/11/24-00:23:09.310607 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285672.sst): old prealloc_size = 147639500, real_size = 32097200 2024/11/24-00:23:09.311185 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285672.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926833 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0301 GB avg-val = 116.81 avg-zval = 32.48 2024/11/24-00:23:09.311925 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285672.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:23:10.143001 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285672.sst): second pass time = 1.47's, 73.829'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.346'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3161.997'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 281.677'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35662.594'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.829'MB/sec zip pipeline throughput = 73.829'MB/sec entries = 926833 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.81 avg-zval = 32.48 usrkeys = 926833 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.81 avg-zval = 32.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926833 bytes = 3128080 } seq expand size = 6487831 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0301 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.5150 value = 3.5966 dict = 1.00 all = 3.7201 } Zip/UnZip{ index = 0.0951 value = 0.2780 dict = 1.00 all = 0.2688 } ---------------------------- total value len = 9.307174 GB avg = 0.166 KB (by entry num) total key len = 1.545243 GB avg = 0.028 KB total ukey len = 1.072465 GB avg = 0.015 KB total ukey num = 0.069279801 Billion total entry num = 0.056087530 Billion write speed all = 0.194914175 MB/sec (with seq num) write speed all = 0.186855322 MB/sec (without seq num) 2024/11/24-00:23:10.189830 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285672: 926833 keys, 36079446 bytes, temperature: kWarm 2024/11/24-00:23:10.189900 139957433374272 EVENT_LOG_v1 {"time_micros": 1732378990189865, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285672, "file_size": 36079446, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4346790653, "table_properties": {"data_size": 30102432, "index_size": 1762880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951324, "raw_average_key_size": 28, "raw_value_size": 108266410, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926833, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378986, "oldest_key_time": 0, "file_creation_time": 1732378986, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285672, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:10.228556 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285673.sst) /dev/shm/ToplingTemp/Topling-225811-eG5JnW 2024/11/24-00:23:11.069656 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285673.sst): first pass time = 0.84's, 159.581'MB/sec 2024/11/24-00:23:12.558236 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285673.sst): old prealloc_size = 147639500, real_size = 33053424 2024/11/24-00:23:12.558768 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285673.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927717 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.68 avg-zval = 33.48 2024/11/24-00:23:12.559536 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285673.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:23:13.415765 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285673.sst): second pass time = 1.49's, 72.756'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.193'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3468.485'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 293.488'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35772.757'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.756'MB/sec zip pipeline throughput = 72.756'MB/sec entries = 927717 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 33.48 usrkeys = 927717 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 33.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927717 bytes = 3131056 } seq expand size = 6494019 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5136 value = 3.4853 dict = 1.00 all = 3.6238 } Zip/UnZip{ index = 0.0951 value = 0.2869 dict = 1.00 all = 0.2760 } ---------------------------- total value len = 9.421910 GB avg = 0.165 KB (by entry num) total key len = 1.571219 GB avg = 0.028 KB total ukey len = 1.091020 GB avg = 0.016 KB total ukey num = 0.070207518 Billion total entry num = 0.057015247 Billion write speed all = 0.197429816 MB/sec (with seq num) write speed all = 0.189238147 MB/sec (without seq num) 2024/11/24-00:23:13.464249 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285673: 927717 keys, 37038260 bytes, temperature: kWarm 2024/11/24-00:23:13.464321 139957433374272 EVENT_LOG_v1 {"time_micros": 1732378993464283, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285673, "file_size": 37038260, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4346788959, "table_properties": {"data_size": 31056480, "index_size": 1764800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976076, "raw_average_key_size": 28, "raw_value_size": 108241710, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927717, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378990, "oldest_key_time": 0, "file_creation_time": 1732378990, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285673, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:13.566068 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285674.sst) /dev/shm/ToplingTemp/Topling-225811-9QMEzZ 2024/11/24-00:23:14.401357 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285674.sst): first pass time = 0.84's, 160.692'MB/sec 2024/11/24-00:23:15.841888 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285674.sst): old prealloc_size = 147639500, real_size = 33126672 2024/11/24-00:23:15.842435 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285674.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929334 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.42 avg-zval = 33.49 2024/11/24-00:23:15.843191 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285674.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:23:16.710367 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285674.sst): second pass time = 1.44's, 75.146'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.738'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3405.298'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 286.614'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 38196.673'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.146'MB/sec zip pipeline throughput = 75.146'MB/sec entries = 929334 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.42 avg-zval = 33.49 usrkeys = 929334 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.42 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929334 bytes = 3136512 } seq expand size = 6505338 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5128 value = 3.4761 dict = 1.00 all = 3.6162 } Zip/UnZip{ index = 0.0951 value = 0.2877 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 9.536612 GB avg = 0.165 KB (by entry num) total key len = 1.597241 GB avg = 0.028 KB total ukey len = 1.109606 GB avg = 0.016 KB total ukey num = 0.071136852 Billion total entry num = 0.057944581 Billion write speed all = 0.199945285 MB/sec (with seq num) write speed all = 0.191620586 MB/sec (without seq num) 2024/11/24-00:23:16.756700 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285674: 929334 keys, 37115744 bytes, temperature: kWarm 2024/11/24-00:23:16.756767 139957433374272 EVENT_LOG_v1 {"time_micros": 1732378996756733, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285674, "file_size": 37115744, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4346790597, "table_properties": {"data_size": 31126144, "index_size": 1768000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26021352, "raw_average_key_size": 28, "raw_value_size": 108196384, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929334, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378993, "oldest_key_time": 0, "file_creation_time": 1732378993, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285674, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:16.794836 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285675.sst) /dev/shm/ToplingTemp/Topling-225811-XYEabZ 2024/11/24-00:23:17.756806 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285675.sst): first pass time = 0.96's, 139.529'MB/sec 2024/11/24-00:23:19.229109 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285675.sst): old prealloc_size = 147639500, real_size = 33100640 2024/11/24-00:23:19.229676 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285675.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928553 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.55 avg-zval = 33.49 2024/11/24-00:23:19.230462 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285675.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:23:20.096603 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285675.sst): second pass time = 1.47's, 73.544'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.754'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3251.731'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.13's, 295.299'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34836.441'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.544'MB/sec zip pipeline throughput = 73.544'MB/sec entries = 928553 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.55 avg-zval = 33.49 usrkeys = 928553 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.55 avg-zval = 33.49 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928553 bytes = 3133888 } seq expand size = 6499871 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5131 value = 3.4795 dict = 1.00 all = 3.6189 } Zip/UnZip{ index = 0.0951 value = 0.2874 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 9.651330 GB avg = 0.164 KB (by entry num) total key len = 1.623240 GB avg = 0.028 KB total ukey len = 1.128177 GB avg = 0.016 KB total ukey num = 0.072065405 Billion total entry num = 0.058873134 Billion write speed all = 0.202460027 MB/sec (with seq num) write speed all = 0.194002441 MB/sec (without seq num) 2024/11/24-00:23:20.143931 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285675: 928553 keys, 37088068 bytes, temperature: kWarm 2024/11/24-00:23:20.144018 139957433374272 EVENT_LOG_v1 {"time_micros": 1732379000143971, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285675, "file_size": 37088068, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4346790485, "table_properties": {"data_size": 31101840, "index_size": 1766464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25999484, "raw_average_key_size": 28, "raw_value_size": 108218386, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928553, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732378996, "oldest_key_time": 0, "file_creation_time": 1732378996, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285675, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:20.181746 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285676.sst) /dev/shm/ToplingTemp/Topling-225811-xdpuaW 2024/11/24-00:23:21.183951 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285676.sst): first pass time = 1.00's, 133.927'MB/sec 2024/11/24-00:23:22.520704 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285676.sst): old prealloc_size = 147639500, real_size = 32729952 2024/11/24-00:23:22.521270 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285676.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928731 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.52 avg-zval = 33.08 2024/11/24-00:23:22.522025 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285676.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:23:23.374803 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285676.sst): second pass time = 1.34's, 81.039'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.991'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3268.505'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 289.572'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32093.455'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.039'MB/sec zip pipeline throughput = 81.039'MB/sec entries = 928731 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.52 avg-zval = 33.08 usrkeys = 928731 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.52 avg-zval = 33.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928731 bytes = 3134480 } seq expand size = 6501117 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.4455 value = 3.5226 dict = 1.00 all = 3.6554 } Zip/UnZip{ index = 0.0957 value = 0.2839 dict = 1.00 all = 0.2736 } ---------------------------- total value len = 9.766044 GB avg = 0.163 KB (by entry num) total key len = 1.649245 GB avg = 0.028 KB total ukey len = 1.146752 GB avg = 0.016 KB total ukey num = 0.072994136 Billion total entry num = 0.059801865 Billion write speed all = 0.204974883 MB/sec (with seq num) write speed all = 0.196384383 MB/sec (without seq num) 2024/11/24-00:23:23.421351 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285676: 928731 keys, 36717656 bytes, temperature: kWarm 2024/11/24-00:23:23.421421 139957433374272 EVENT_LOG_v1 {"time_micros": 1732379003421384, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285676, "file_size": 36717656, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4346790550, "table_properties": {"data_size": 30719376, "index_size": 1778240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26004468, "raw_average_key_size": 28, "raw_value_size": 108213362, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928731, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379000, "oldest_key_time": 0, "file_creation_time": 1732379000, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285676, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:23.459686 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285677.sst) /dev/shm/ToplingTemp/Topling-225811-G46S1X 2024/11/24-00:23:24.478765 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285677.sst): first pass time = 1.02's, 131.710'MB/sec 2024/11/24-00:23:25.780899 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285677.sst): old prealloc_size = 147639500, real_size = 31984464 2024/11/24-00:23:25.781437 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285677.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929049 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0300 GB avg-val = 116.47 avg-zval = 32.27 2024/11/24-00:23:25.782241 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285677.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:23:26.614162 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285677.sst): second pass time = 1.30's, 83.150'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.173'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3417.974'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.12's, 287.991'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35228.470'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.150'MB/sec zip pipeline throughput = 83.150'MB/sec entries = 929049 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.47 avg-zval = 32.27 usrkeys = 929049 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.47 avg-zval = 32.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929049 bytes = 3135552 } seq expand size = 6503343 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5130 value = 3.6087 dict = 1.00 all = 3.7311 } Zip/UnZip{ index = 0.0951 value = 0.2771 dict = 1.00 all = 0.2680 } ---------------------------- total value len = 9.880752 GB avg = 0.163 KB (by entry num) total key len = 1.675258 GB avg = 0.028 KB total ukey len = 1.165333 GB avg = 0.016 KB total ukey num = 0.073923185 Billion total entry num = 0.060730914 Billion write speed all = 0.207489626 MB/sec (with seq num) write speed all = 0.198766176 MB/sec (without seq num) 2024/11/24-00:23:26.659063 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285677: 929049 keys, 35973090 bytes, temperature: kWarm 2024/11/24-00:23:26.659126 139957433374272 EVENT_LOG_v1 {"time_micros": 1732379006659091, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285677, "file_size": 35973090, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4346789703, "table_properties": {"data_size": 29984576, "index_size": 1767424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26013372, "raw_average_key_size": 28, "raw_value_size": 108204382, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929049, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379003, "oldest_key_time": 0, "file_creation_time": 1732379003, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285677, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:26.693645 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285678.sst) /dev/shm/ToplingTemp/Topling-225811-c59XYW 2024/11/24-00:23:27.668396 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285678.sst): first pass time = 0.97's, 137.699'MB/sec 2024/11/24-00:23:28.992077 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285678.sst): old prealloc_size = 147639500, real_size = 30862016 2024/11/24-00:23:28.992578 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285678.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928788 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0289 GB avg-val = 116.51 avg-zval = 31.06 2024/11/24-00:23:28.993339 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285678.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:23:29.809333 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285678.sst): second pass time = 1.32's, 81.837'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.644'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3703.358'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.12's, 279.250'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34574.649'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.837'MB/sec zip pipeline throughput = 81.837'MB/sec entries = 928788 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.51 avg-zval = 31.06 usrkeys = 928788 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.51 avg-zval = 31.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928788 bytes = 3134672 } seq expand size = 6501516 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0348 GB } UnZip/Zip{ index = 10.4443 value = 3.7507 dict = 1.00 all = 3.8513 } Zip/UnZip{ index = 0.0957 value = 0.2666 dict = 1.00 all = 0.2597 } ---------------------------- total value len = 9.995465 GB avg = 0.162 KB (by entry num) total key len = 1.701264 GB avg = 0.028 KB total ukey len = 1.183909 GB avg = 0.016 KB total ukey num = 0.074851973 Billion total entry num = 0.061659702 Billion write speed all = 0.210004212 MB/sec (with seq num) write speed all = 0.201147858 MB/sec (without seq num) 2024/11/24-00:23:29.854229 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285678: 928788 keys, 34849756 bytes, temperature: kWarm 2024/11/24-00:23:29.854299 139957433374272 EVENT_LOG_v1 {"time_micros": 1732379009854264, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285678, "file_size": 34849756, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4346789720, "table_properties": {"data_size": 28851056, "index_size": 1778560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26006064, "raw_average_key_size": 28, "raw_value_size": 108211792, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928788, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379006, "oldest_key_time": 0, "file_creation_time": 1732379006, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285678, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:29.890869 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285679.sst) /dev/shm/ToplingTemp/Topling-225811-ZO9bFY 2024/11/24-00:23:30.967455 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285679.sst): first pass time = 1.08's, 124.674'MB/sec 2024/11/24-00:23:32.249143 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285679.sst): old prealloc_size = 147639500, real_size = 30668512 2024/11/24-00:23:32.249701 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285679.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926489 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0287 GB avg-val = 116.87 avg-zval = 30.94 2024/11/24-00:23:32.250469 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285679.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:23:33.065129 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285679.sst): second pass time = 1.28's, 84.550'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.463'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3318.812'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 275.344'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34632.864'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.550'MB/sec zip pipeline throughput = 84.550'MB/sec entries = 926489 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.87 avg-zval = 30.94 usrkeys = 926489 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.87 avg-zval = 30.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926489 bytes = 3126912 } seq expand size = 6485423 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4443 value = 3.7776 dict = 1.00 all = 3.8737 } Zip/UnZip{ index = 0.0957 value = 0.2647 dict = 1.00 all = 0.2582 } ---------------------------- total value len = 10.110227 GB avg = 0.162 KB (by entry num) total key len = 1.727206 GB avg = 0.028 KB total ukey len = 1.202439 GB avg = 0.016 KB total ukey num = 0.075778462 Billion total entry num = 0.062586191 Billion write speed all = 0.212517988 MB/sec (with seq num) write speed all = 0.203529085 MB/sec (without seq num) 2024/11/24-00:23:33.107772 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285679: 926489 keys, 34648340 bytes, temperature: kWarm 2024/11/24-00:23:33.107843 139957433374272 EVENT_LOG_v1 {"time_micros": 1732379013107805, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285679, "file_size": 34648340, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4269403493, "largest_seqno": 4346789731, "table_properties": {"data_size": 28662544, "index_size": 1774144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25941692, "raw_average_key_size": 28, "raw_value_size": 108276042, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926489, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379009, "oldest_key_time": 0, "file_creation_time": 1732379009, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285679, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:33.148025 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285680.sst) /dev/shm/ToplingTemp/Topling-225811-8retuY 2024/11/24-00:23:34.127600 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285680.sst): first pass time = 0.98's, 137.021'MB/sec 2024/11/24-00:23:35.440082 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285680.sst): old prealloc_size = 147639500, real_size = 31812928 2024/11/24-00:23:35.440592 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285680.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925514 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0298 GB avg-val = 117.02 avg-zval = 32.22 2024/11/24-00:23:35.441362 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285680.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-00:23:36.261882 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285680.sst): second pass time = 1.31's, 82.568'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.554'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3600.748'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 285.443'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31719.687'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.568'MB/sec zip pipeline throughput = 82.568'MB/sec entries = 925514 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.02 avg-zval = 32.22 usrkeys = 925514 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.02 avg-zval = 32.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925514 bytes = 3123632 } seq expand size = 6478598 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5149 value = 3.6318 dict = 1.00 all = 3.7500 } Zip/UnZip{ index = 0.0951 value = 0.2753 dict = 1.00 all = 0.2667 } ---------------------------- total value len = 10.225009 GB avg = 0.161 KB (by entry num) total key len = 1.753120 GB avg = 0.028 KB total ukey len = 1.220949 GB avg = 0.016 KB total ukey num = 0.076703976 Billion total entry num = 0.063511705 Billion write speed all = 0.215031577 MB/sec (with seq num) write speed all = 0.205910271 MB/sec (without seq num) 2024/11/24-00:23:36.308563 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285680: 925514 keys, 35791650 bytes, temperature: kWarm 2024/11/24-00:23:36.308634 139957433374272 EVENT_LOG_v1 {"time_micros": 1732379016308596, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285680, "file_size": 35791650, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679976, "largest_seqno": 4346789797, "table_properties": {"data_size": 29820976, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25914392, "raw_average_key_size": 28, "raw_value_size": 108303472, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925514, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379013, "oldest_key_time": 0, "file_creation_time": 1732379013, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285680, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:36.345371 139957433374272 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285681.sst) /dev/shm/ToplingTemp/Topling-225811-3tREaX 2024/11/24-00:23:36.459675 139957433374272 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285681.sst): first pass time = 0.11's, 243.253'MB/sec 2024/11/24-00:23:36.717870 139957433374272 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285681.sst): old prealloc_size = 147639500, real_size = 7313616 2024/11/24-00:23:36.718060 139957433374272 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285681.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 195743 prefix = 4 raw-key = 0.0031 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0223 GB zip-val = 0.0069 GB avg-val = 114.00 avg-zval = 35.18 2024/11/24-00:23:36.718251 139957433374272 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285681.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.028 GB 2024/11/24-00:23:36.890062 139957433374272 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00/285681.sst): second pass time = 0.26's, 86.631'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 47.110'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2180.687'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.337'MB/sec write SST data time = 0.03's, 304.585'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.233'MB, throughput = 7591.569'MB/sec zip my value time = 0.26's, unzip length = 0.022'GB zip my value throughput = 86.631'MB/sec zip pipeline throughput = 86.631'MB/sec entries = 195743 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.00 avg-zval = 35.18 usrkeys = 195743 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.00 avg-zval = 35.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 195743 bytes = 611712 } seq expand size = 1370201 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0039 GB value = 0.0223 GB dict = 0.23 MB all = 0.0278 GB } __ZipSize{ index = 0.0004 GB value = 0.0069 GB dict = 0.23 MB all = 0.0081 GB } UnZip/Zip{ index = 10.3397 value = 3.2406 dict = 1.00 all = 3.4269 } Zip/UnZip{ index = 0.0967 value = 0.3086 dict = 1.00 all = 0.2918 } ---------------------------- total value len = 10.248694 GB avg = 0.161 KB (by entry num) total key len = 1.758601 GB avg = 0.028 KB total ukey len = 1.224864 GB avg = 0.016 KB total ukey num = 0.076899719 Billion total entry num = 0.063707448 Billion write speed all = 0.215552730 MB/sec (with seq num) write speed all = 0.206403415 MB/sec (without seq num) 2024/11/24-00:23:36.905573 139957433374272 [compaction_job.cc:1948] [default] [JOB 38509] Generated table #285681: 195743 keys, 8110913 bytes, temperature: kWarm 2024/11/24-00:23:36.905628 139957433374272 EVENT_LOG_v1 {"time_micros": 1732379016905598, "cf_name": "default", "job": 38509, "event": "table_file_creation", "file_number": 285681, "file_size": 8110913, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4325524821, "largest_seqno": 4346790849, "table_properties": {"data_size": 6885888, "index_size": 378624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 5480804, "raw_average_key_size": 28, "raw_value_size": 22314702, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 195743, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732379016, "oldest_key_time": 0, "file_creation_time": 1732379016, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 285681, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-00:23:36.917484 139957433374272 [compaction_job.cc:703] job-38509: subcompact[0], size: 0.439451 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-38509/att-00 [285669,285670,285671,285672,285673,285674,285675,285676,285677,285678,285679,285680,285681] 2024/11/24-00:23:36.995941 139957433374272 (Original Log Time 2024/11/24-00:22:53.643114) EVENT_LOG_v1 {"time_micros": 1732378973643073, "job": 38509, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [285666, 285661], "files_L3": [285623, 285624, 285625, 285626, 285627, 285628, 285629, 285630, 285631, 285632, 285633, 285604, 285605], "score": 0.465089, "input_data_size": 467264836, "oldest_snapshot_seqno": 4346804510} 2024/11/24-00:23:36.995945 139957433374272 (Original Log Time 2024/11/24-00:23:36.994819) [compaction_job.cc:2022] [default] [JOB 38509] Compacted 2@2 + 13@3 files to L3 => 439451447 bytes 2024/11/24-00:23:36.995946 139957433374272 (Original Log Time 2024/11/24-00:23:36.995876) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 10.8 rd, 10.2 wr, level 3, files in(2, 13) out(13 +0 blob) MB in(29.6, 416.0 +0.0 blob) out(419.1 +0.0 blob), read-write-amplify(29.2) write-amplify(14.2) OK, records in: 11448184, records dropped: 98900 output_compression: Snappy 2024/11/24-00:23:36.995951 139957433374272 (Original Log Time 2024/11/24-00:23:36.995920) EVENT_LOG_v1 {"time_micros": 1732379016995900, "cf": "default", "job": 38509, "event": "compaction_finished", "compaction_time_micros": 43274374, "compaction_time_cpu_micros": 18481789, "output_level": 3, "num_output_files": 13, "total_output_size": 439451447, "num_input_records": 11448184, "num_output_records": 11349284, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 320620573, "file_range_sync_nanos": 0, "file_fsync_nanos": 10234, "file_prepare_write_nanos": 33589, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/24-00:23:37.120148 139957433374272 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732378973,"executesMs":43478,"compactionJobId":38509,"attempt":0,"compactionInputRawBytes":1652452772,"compactionInputZipBytes":467264836,"compactionOutputRawBytes":0,"compactionOutputZipBytes":439451447,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-268b6741dbe96f561c494dfc2bec9239-2c1ad065a309b6cc-00"}}, response = 2024/11/24-00:23:37.120234 139957433374272 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-38509/att-00: olev 3, work 43.479 s, result 7.751 ms, install 1.147 ms, input{raw 1.539 GiB zip 445.618 MiB}