2024/11/24-06:02:36.541162 139957676762688 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00: fork to child time = 0.008233 sec 2024/11/24-06:02:36.542052 139957676762688 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00 2024/11/24-06:02:36.549437 139957676762688 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00 2024/11/24-06:02:36.550393 139957676762688 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-06:02:36.550629 139957676762688 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-06:02:36.610616 139957676762688 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42554/att-00/MANIFEST-206347 2024/11/24-06:02:36.629896 139957676762688 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42554/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 294349, last_sequence is 4470961311, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 294213 2024/11/24-06:02:36.629904 139957676762688 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-06:02:36.630079 139957676762688 [version_set.cc:5721] Creating manifest 294349 2024/11/24-06:02:36.665460 139957676762688 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-06:02:36.687660 139957676762688 [compaction_job.cc:2437] [default] [JOB 42554] Compacting 1@2 + 23@3 files to L3, score 0.94, subcompactions 1 : 1 2024/11/24-06:02:36.687706 139957676762688 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31904 Base level 2, inputs: [294346(27MB)], [294271(34MB) 294272(33MB) 294273(35MB) 294274(33MB) 294275(35MB) 294312(34MB) 294313(33MB) 294314(35MB) 294315(33MB) 294316(35MB) 294317(33MB) 294318(33MB) 294319(35MB) 294320(33MB) 294321(35MB) 294322(33MB) 294323(10MB) 294284(33MB) 294285(35MB) 294286(33MB) 294287(34MB) 294288(30MB) 294289(32MB)] 2024/11/24-06:02:36.694456 139957676762688 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:02:36.694482 139957676762688 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:02:36.694498 139957676762688 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:02:36.694504 139957676762688 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-06:02:36.694515 139957676762688 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-06:02:37.268684 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294350.sst) /dev/shm/ToplingTemp/Topling-285124-kWIzjW 2024/11/24-06:02:37.868165 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294350.sst): first pass time = 0.60's, 223.904'MB/sec 2024/11/24-06:02:39.466468 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294350.sst): old prealloc_size = 147639500, real_size = 32353584 2024/11/24-06:02:39.467202 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294350.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927743 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0304 GB avg-val = 116.67 avg-zval = 32.72 2024/11/24-06:02:39.468108 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294350.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:02:40.567177 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294350.sst): second pass time = 1.60's, 67.762'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 33.120'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2476.934'MB/sec (index lex order gen) rebuild zvType time = 0.94's, 0.247'MB/sec write SST data time = 0.16's, 225.445'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34004.596'MB/sec zip my value time = 1.60's, unzip length = 0.108'GB zip my value throughput = 67.762'MB/sec zip pipeline throughput = 67.762'MB/sec entries = 927743 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.67 avg-zval = 32.72 usrkeys = 927743 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.67 avg-zval = 32.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927743 bytes = 3247120 } seq expand size = 6494201 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5120 value = 3.5657 dict = 1.00 all = 3.6818 } Zip/UnZip{ index = 0.0951 value = 0.2805 dict = 1.00 all = 0.2716 } ---------------------------- total value len = 2251.440308 GB avg = 0.200 KB (by entry num) total key len = 306.908939 GB avg = 0.027 KB total ukey len = 237.373282 GB avg = 0.015 KB total ukey num = 15.739588906 Billion total entry num = 11.280902854 Billion write speed all = 33.641087055 MB/sec (with seq num) write speed all = 32.454378601 MB/sec (without seq num) 2024/11/24-06:02:40.615535 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294350: 927743 keys, 36454832 bytes, temperature: kWarm 2024/11/24-06:02:40.615621 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399360615568, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294350, "file_size": 36454832, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4470946804, "table_properties": {"data_size": 30356320, "index_size": 1765120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976804, "raw_average_key_size": 28, "raw_value_size": 108241018, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927743, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399357, "oldest_key_time": 0, "file_creation_time": 1732399357, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294350, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:02:40.663676 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294351.sst) /dev/shm/ToplingTemp/Topling-285124-Q0ILZX 2024/11/24-06:02:41.672032 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294351.sst): first pass time = 1.01's, 133.110'MB/sec 2024/11/24-06:02:43.885547 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294351.sst): old prealloc_size = 147639500, real_size = 31298944 2024/11/24-06:02:43.886266 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294351.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.88 2024/11/24-06:02:43.887201 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294351.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:02:44.803520 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294351.sst): second pass time = 2.21's, 49.042'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 38.538'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2523.055'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.303'MB/sec write SST data time = 0.16's, 224.508'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34839.611'MB/sec zip my value time = 2.21's, unzip length = 0.108'GB zip my value throughput = 49.042'MB/sec zip pipeline throughput = 58.036'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.88 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4447 value = 3.7012 dict = 1.00 all = 3.7941 } Zip/UnZip{ index = 0.0957 value = 0.2702 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2251.555221 GB avg = 0.200 KB (by entry num) total key len = 306.934680 GB avg = 0.027 KB total ukey len = 237.391668 GB avg = 0.015 KB total ukey num = 15.740508206 Billion total entry num = 11.281822154 Billion write speed all = 33.641062572 MB/sec (with seq num) write speed all = 32.454323519 MB/sec (without seq num) 2024/11/24-06:02:44.896538 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294351: 919300 keys, 35375666 bytes, temperature: kWarm 2024/11/24-06:02:44.896607 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399364896572, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294351, "file_size": 35375666, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4289749651, "largest_seqno": 4441748665, "table_properties": {"data_size": 29308592, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399360, "oldest_key_time": 0, "file_creation_time": 1732399360, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294351, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:02:44.940516 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294352.sst) /dev/shm/ToplingTemp/Topling-285124-ig2odV 2024/11/24-06:02:45.566128 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294352.sst): first pass time = 0.63's, 112.466'MB/sec 2024/11/24-06:02:46.842903 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294352.sst): old prealloc_size = 147639500, real_size = 17188128 2024/11/24-06:02:46.843253 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294352.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 490352 prefix = 4 raw-key = 0.0078 GB zip-key = 0.0009 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0566 GB zip-val = 0.0161 GB avg-val = 115.48 avg-zval = 32.87 2024/11/24-06:02:46.843672 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294352.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0012 GB, waited 0.000 sec, Key+Value = 0.070 GB 2024/11/24-06:02:47.295870 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294352.sst): second pass time = 1.28's, 44.375'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.45's, 42.450'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2867.227'MB/sec (index lex order gen) rebuild zvType time = 0.39's, 0.318'MB/sec write SST data time = 0.07's, 286.519'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.565'MB, throughput = 14467.707'MB/sec zip my value time = 1.28's, unzip length = 0.057'GB zip my value throughput = 44.375'MB/sec zip pipeline throughput = 87.628'MB/sec entries = 490352 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.48 avg-zval = 32.87 usrkeys = 490352 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.48 avg-zval = 32.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 490352 bytes = 1593664 } seq expand size = 3432464 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0098 GB value = 0.0566 GB dict = 0.57 MB all = 0.0704 GB } __ZipSize{ index = 0.0009 GB value = 0.0161 GB dict = 0.57 MB all = 0.0192 GB } UnZip/Zip{ index = 10.3712 value = 3.5128 dict = 1.00 all = 3.6592 } Zip/UnZip{ index = 0.0964 value = 0.2847 dict = 1.00 all = 0.2733 } ---------------------------- total value len = 2251.615279 GB avg = 0.200 KB (by entry num) total key len = 306.948410 GB avg = 0.027 KB total ukey len = 237.401475 GB avg = 0.015 KB total ukey num = 15.740998558 Billion total entry num = 11.282312506 Billion write speed all = 33.640930336 MB/sec (with seq num) write speed all = 32.454178595 MB/sec (without seq num) 2024/11/24-06:02:47.323922 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294352: 490352 keys, 19227365 bytes, temperature: kWarm 2024/11/24-06:02:47.323996 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399367323951, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294352, "file_size": 19227365, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4425679736, "largest_seqno": 4470947562, "table_properties": {"data_size": 16119760, "index_size": 945600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 13729856, "raw_average_key_size": 28, "raw_value_size": 56626004, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 490352, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399364, "oldest_key_time": 0, "file_creation_time": 1732399364, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294352, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:02:47.353157 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294353.sst) /dev/shm/ToplingTemp/Topling-285124-dJdavW 2024/11/24-06:02:48.349435 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294353.sst): first pass time = 1.00's, 134.724'MB/sec 2024/11/24-06:02:49.658906 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294353.sst): old prealloc_size = 147639500, real_size = 31398672 2024/11/24-06:02:49.659508 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294353.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.99 2024/11/24-06:02:49.660284 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294353.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:02:50.472673 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294353.sst): second pass time = 1.31's, 82.916'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.596'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3041.458'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 278.561'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34180.003'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.916'MB/sec zip pipeline throughput = 82.916'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.99 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6886 dict = 1.00 all = 3.7834 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2251.730192 GB avg = 0.200 KB (by entry num) total key len = 306.974150 GB avg = 0.027 KB total ukey len = 237.419861 GB avg = 0.015 KB total ukey num = 15.741917858 Billion total entry num = 11.283231806 Billion write speed all = 33.641374507 MB/sec (with seq num) write speed all = 32.454575640 MB/sec (without seq num) 2024/11/24-06:02:50.516670 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294353: 919300 keys, 35475378 bytes, temperature: kWarm 2024/11/24-06:02:50.516738 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399370516703, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294353, "file_size": 35475378, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4414357721, "table_properties": {"data_size": 29408512, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399367, "oldest_key_time": 0, "file_creation_time": 1732399367, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294353, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:02:50.556516 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294354.sst) /dev/shm/ToplingTemp/Topling-285124-4X1kHV 2024/11/24-06:02:51.497852 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294354.sst): first pass time = 0.94's, 142.587'MB/sec 2024/11/24-06:02:52.844566 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294354.sst): old prealloc_size = 147639500, real_size = 32931184 2024/11/24-06:02:52.845142 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294354.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927716 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.68 avg-zval = 33.34 2024/11/24-06:02:52.845913 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294354.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:02:53.692053 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294354.sst): second pass time = 1.35's, 80.457'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.698'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.687'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 287.919'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35932.409'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.457'MB/sec zip pipeline throughput = 80.457'MB/sec entries = 927716 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 33.34 usrkeys = 927716 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 33.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927716 bytes = 3247024 } seq expand size = 6494012 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5116 value = 3.4991 dict = 1.00 all = 3.6243 } Zip/UnZip{ index = 0.0951 value = 0.2858 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 2251.844928 GB avg = 0.200 KB (by entry num) total key len = 307.000126 GB avg = 0.027 KB total ukey len = 237.438416 GB avg = 0.015 KB total ukey num = 15.742845574 Billion total entry num = 11.284159522 Billion write speed all = 33.641800571 MB/sec (with seq num) write speed all = 32.454954361 MB/sec (without seq num) 2024/11/24-06:02:53.737623 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294354: 927716 keys, 37032316 bytes, temperature: kWarm 2024/11/24-06:02:53.737698 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399373737657, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294354, "file_size": 37032316, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178655, "largest_seqno": 4470947713, "table_properties": {"data_size": 30933920, "index_size": 1765120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976048, "raw_average_key_size": 28, "raw_value_size": 108241820, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927716, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399370, "oldest_key_time": 0, "file_creation_time": 1732399370, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294354, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:02:53.774232 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294355.sst) /dev/shm/ToplingTemp/Topling-285124-AcW3EY 2024/11/24-06:02:54.784457 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294355.sst): first pass time = 1.01's, 132.864'MB/sec 2024/11/24-06:02:56.127226 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294355.sst): old prealloc_size = 147639500, real_size = 31383008 2024/11/24-06:02:56.127802 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294355.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.97 2024/11/24-06:02:56.128589 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294355.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:02:56.941748 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294355.sst): second pass time = 1.34's, 80.836'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.536'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3187.755'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 275.263'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34272.854'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.836'MB/sec zip pipeline throughput = 80.836'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.97 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4455 value = 3.6906 dict = 1.00 all = 3.7851 } Zip/UnZip{ index = 0.0957 value = 0.2710 dict = 1.00 all = 0.2642 } ---------------------------- total value len = 2251.959840 GB avg = 0.200 KB (by entry num) total key len = 307.025866 GB avg = 0.027 KB total ukey len = 237.456802 GB avg = 0.015 KB total ukey num = 15.743764874 Billion total entry num = 11.285078822 Billion write speed all = 33.642212422 MB/sec (with seq num) write speed all = 32.455320231 MB/sec (without seq num) 2024/11/24-06:02:56.984604 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294355: 919300 keys, 35459698 bytes, temperature: kWarm 2024/11/24-06:02:56.984672 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399376984637, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294355, "file_size": 35459698, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4307516599, "largest_seqno": 4458712729, "table_properties": {"data_size": 29392784, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399373, "oldest_key_time": 0, "file_creation_time": 1732399373, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294355, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:02:57.019185 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294356.sst) /dev/shm/ToplingTemp/Topling-285124-undmDX 2024/11/24-06:02:58.027277 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294356.sst): first pass time = 1.01's, 133.145'MB/sec 2024/11/24-06:02:59.350810 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294356.sst): old prealloc_size = 147639500, real_size = 33320896 2024/11/24-06:02:59.351381 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294356.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927447 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.72 avg-zval = 33.77 2024/11/24-06:02:59.352166 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294356.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:00.217129 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294356.sst): second pass time = 1.32's, 81.858'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.197'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3230.662'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 289.161'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31082.492'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.858'MB/sec zip pipeline throughput = 81.858'MB/sec entries = 927447 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.72 avg-zval = 33.77 usrkeys = 927447 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.72 avg-zval = 33.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927447 bytes = 3246080 } seq expand size = 6492129 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5109 value = 3.4558 dict = 1.00 all = 3.5867 } Zip/UnZip{ index = 0.0951 value = 0.2894 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 2252.074582 GB avg = 0.200 KB (by entry num) total key len = 307.051835 GB avg = 0.027 KB total ukey len = 237.475350 GB avg = 0.015 KB total ukey num = 15.744692321 Billion total entry num = 11.286006269 Billion write speed all = 33.642613627 MB/sec (with seq num) write speed all = 32.455675003 MB/sec (without seq num) 2024/11/24-06:03:00.262078 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294356: 927447 keys, 37421280 bytes, temperature: kWarm 2024/11/24-06:03:00.262148 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399380262112, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294356, "file_size": 37421280, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4470947058, "table_properties": {"data_size": 31324144, "index_size": 1764736, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25968516, "raw_average_key_size": 28, "raw_value_size": 108249330, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927447, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399377, "oldest_key_time": 0, "file_creation_time": 1732399377, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294356, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:00.310663 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294357.sst) /dev/shm/ToplingTemp/Topling-285124-JIsJoV 2024/11/24-06:03:01.337222 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294357.sst): first pass time = 1.03's, 130.750'MB/sec 2024/11/24-06:03:02.664841 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294357.sst): old prealloc_size = 147639500, real_size = 30901424 2024/11/24-06:03:02.665399 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294357.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925192 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.07 avg-zval = 31.23 2024/11/24-06:03:02.666156 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294357.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:03.479663 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294357.sst): second pass time = 1.33's, 81.650'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.805'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3282.349'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 278.071'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31005.278'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.650'MB/sec zip pipeline throughput = 81.650'MB/sec entries = 925192 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.07 avg-zval = 31.23 usrkeys = 925192 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.07 avg-zval = 31.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925192 bytes = 3122544 } seq expand size = 6476344 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4437 value = 3.7481 dict = 1.00 all = 3.8483 } Zip/UnZip{ index = 0.0958 value = 0.2668 dict = 1.00 all = 0.2599 } ---------------------------- total value len = 2252.189370 GB avg = 0.200 KB (by entry num) total key len = 307.077740 GB avg = 0.027 KB total ukey len = 237.493854 GB avg = 0.015 KB total ukey num = 15.745617513 Billion total entry num = 11.286931461 Billion write speed all = 33.643020270 MB/sec (with seq num) write speed all = 32.456035254 MB/sec (without seq num) 2024/11/24-06:03:03.523151 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294357: 925192 keys, 34877592 bytes, temperature: kWarm 2024/11/24-06:03:03.523224 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399383523184, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294357, "file_size": 34877592, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4354613673, "largest_seqno": 4470947072, "table_properties": {"data_size": 28898144, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25905376, "raw_average_key_size": 28, "raw_value_size": 108312380, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925192, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399380, "oldest_key_time": 0, "file_creation_time": 1732399380, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294357, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:03.563924 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294358.sst) /dev/shm/ToplingTemp/Topling-285124-2cShTV 2024/11/24-06:03:04.566262 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294358.sst): first pass time = 1.00's, 133.910'MB/sec 2024/11/24-06:03:05.867922 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294358.sst): old prealloc_size = 147639500, real_size = 32325856 2024/11/24-06:03:05.868515 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294358.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921694 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0303 GB avg-val = 117.62 avg-zval = 32.92 2024/11/24-06:03:05.870586 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294358.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:06.699674 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294358.sst): second pass time = 1.30's, 83.338'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.777'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3077.707'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 281.991'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 30787.884'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.338'MB/sec zip pipeline throughput = 83.338'MB/sec entries = 921694 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.62 avg-zval = 32.92 usrkeys = 921694 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.62 avg-zval = 32.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921694 bytes = 3225936 } seq expand size = 6451858 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5124 value = 3.5730 dict = 1.00 all = 3.6862 } Zip/UnZip{ index = 0.0951 value = 0.2799 dict = 1.00 all = 0.2713 } ---------------------------- total value len = 2252.304233 GB avg = 0.200 KB (by entry num) total key len = 307.103548 GB avg = 0.027 KB total ukey len = 237.512288 GB avg = 0.015 KB total ukey num = 15.746539207 Billion total entry num = 11.287853155 Billion write speed all = 33.643445364 MB/sec (with seq num) write speed all = 32.456413664 MB/sec (without seq num) 2024/11/24-06:03:06.743783 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294358: 921694 keys, 36410702 bytes, temperature: kWarm 2024/11/24-06:03:06.743852 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399386743818, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294358, "file_size": 36410702, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4470947487, "table_properties": {"data_size": 30341712, "index_size": 1753536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25807432, "raw_average_key_size": 28, "raw_value_size": 108410296, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921694, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399383, "oldest_key_time": 0, "file_creation_time": 1732399383, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294358, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:06.782892 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294359.sst) /dev/shm/ToplingTemp/Topling-285124-KXCmNV 2024/11/24-06:03:07.675184 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294359.sst): first pass time = 0.89's, 150.425'MB/sec 2024/11/24-06:03:09.011859 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294359.sst): old prealloc_size = 147639500, real_size = 32538352 2024/11/24-06:03:09.012446 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294359.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927646 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.69 avg-zval = 32.92 2024/11/24-06:03:09.013229 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294359.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:09.853951 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294359.sst): second pass time = 1.34's, 81.029'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.511'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3118.825'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.324'MB/sec write SST data time = 0.13's, 289.622'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34714.244'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.029'MB/sec zip pipeline throughput = 81.029'MB/sec entries = 927646 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.69 avg-zval = 32.92 usrkeys = 927646 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.69 avg-zval = 32.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927646 bytes = 3246768 } seq expand size = 6493522 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5135 value = 3.5441 dict = 1.00 all = 3.6632 } Zip/UnZip{ index = 0.0951 value = 0.2822 dict = 1.00 all = 0.2730 } ---------------------------- total value len = 2252.418970 GB avg = 0.200 KB (by entry num) total key len = 307.129522 GB avg = 0.027 KB total ukey len = 237.530841 GB avg = 0.015 KB total ukey num = 15.747466853 Billion total entry num = 11.288780801 Billion write speed all = 33.643900039 MB/sec (with seq num) write speed all = 32.456820008 MB/sec (without seq num) 2024/11/24-06:03:09.903450 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294359: 927646 keys, 36639398 bytes, temperature: kWarm 2024/11/24-06:03:09.903522 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399389903484, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294359, "file_size": 36639398, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4470947087, "table_properties": {"data_size": 30541600, "index_size": 1764672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25974088, "raw_average_key_size": 28, "raw_value_size": 108243752, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927646, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399386, "oldest_key_time": 0, "file_creation_time": 1732399386, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294359, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:09.942270 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294360.sst) /dev/shm/ToplingTemp/Topling-285124-bU46kU 2024/11/24-06:03:10.897413 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294360.sst): first pass time = 0.96's, 140.527'MB/sec 2024/11/24-06:03:12.219702 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294360.sst): old prealloc_size = 147639500, real_size = 31498320 2024/11/24-06:03:12.220250 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294360.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.10 2024/11/24-06:03:12.221027 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294360.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:13.040780 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294360.sst): second pass time = 1.32's, 82.100'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.329'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3339.586'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 277.643'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32268.915'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.100'MB/sec zip pipeline throughput = 82.100'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.10 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4455 value = 3.6762 dict = 1.00 all = 3.7728 } Zip/UnZip{ index = 0.0957 value = 0.2720 dict = 1.00 all = 0.2651 } ---------------------------- total value len = 2252.533882 GB avg = 0.200 KB (by entry num) total key len = 307.155262 GB avg = 0.027 KB total ukey len = 237.549227 GB avg = 0.015 KB total ukey num = 15.748386153 Billion total entry num = 11.289700101 Billion write speed all = 33.644339513 MB/sec (with seq num) write speed all = 32.457212540 MB/sec (without seq num) 2024/11/24-06:03:13.089514 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294360: 919300 keys, 35575026 bytes, temperature: kWarm 2024/11/24-06:03:13.089584 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399393089548, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294360, "file_size": 35575026, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4293635655, "largest_seqno": 4445393374, "table_properties": {"data_size": 29508096, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399389, "oldest_key_time": 0, "file_creation_time": 1732399389, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294360, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:13.128031 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294361.sst) /dev/shm/ToplingTemp/Topling-285124-woKnHX 2024/11/24-06:03:14.130471 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294361.sst): first pass time = 1.00's, 133.896'MB/sec 2024/11/24-06:03:15.491360 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294361.sst): old prealloc_size = 147639500, real_size = 33342336 2024/11/24-06:03:15.491928 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294361.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927707 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.68 avg-zval = 33.79 2024/11/24-06:03:15.492730 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294361.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:16.357857 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294361.sst): second pass time = 1.36's, 79.614'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.213'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3237.742'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 294.126'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27701.333'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.614'MB/sec zip pipeline throughput = 79.614'MB/sec entries = 927707 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.68 avg-zval = 33.79 usrkeys = 927707 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.68 avg-zval = 33.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927707 bytes = 3246992 } seq expand size = 6493949 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5115 value = 3.4532 dict = 1.00 all = 3.5845 } Zip/UnZip{ index = 0.0951 value = 0.2896 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 2252.648618 GB avg = 0.200 KB (by entry num) total key len = 307.181238 GB avg = 0.027 KB total ukey len = 237.567781 GB avg = 0.015 KB total ukey num = 15.749313860 Billion total entry num = 11.290627808 Billion write speed all = 33.644722128 MB/sec (with seq num) write speed all = 32.457549366 MB/sec (without seq num) 2024/11/24-06:03:16.406448 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294361: 927707 keys, 37443632 bytes, temperature: kWarm 2024/11/24-06:03:16.406519 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399396406483, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294361, "file_size": 37443632, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4470947662, "table_properties": {"data_size": 31345136, "index_size": 1765120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25975796, "raw_average_key_size": 28, "raw_value_size": 108241990, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927707, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399393, "oldest_key_time": 0, "file_creation_time": 1732399393, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294361, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:16.443074 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294362.sst) /dev/shm/ToplingTemp/Topling-285124-OnUriX 2024/11/24-06:03:17.350715 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294362.sst): first pass time = 0.91's, 147.881'MB/sec 2024/11/24-06:03:18.671634 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294362.sst): old prealloc_size = 147639500, real_size = 31114368 2024/11/24-06:03:18.672210 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294362.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922615 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.48 avg-zval = 31.55 2024/11/24-06:03:18.672956 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294362.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:19.486381 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294362.sst): second pass time = 1.32's, 82.140'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.063'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3207.590'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 274.374'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34279.909'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.140'MB/sec zip pipeline throughput = 82.140'MB/sec entries = 922615 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.48 avg-zval = 31.55 usrkeys = 922615 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.48 avg-zval = 31.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922615 bytes = 3113840 } seq expand size = 6458305 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.3760 value = 3.7239 dict = 1.00 all = 3.8256 } Zip/UnZip{ index = 0.0964 value = 0.2685 dict = 1.00 all = 0.2614 } ---------------------------- total value len = 2252.763461 GB avg = 0.200 KB (by entry num) total key len = 307.207071 GB avg = 0.027 KB total ukey len = 237.586234 GB avg = 0.015 KB total ukey num = 15.750236475 Billion total entry num = 11.291550423 Billion write speed all = 33.645187617 MB/sec (with seq num) write speed all = 32.457966663 MB/sec (without seq num) 2024/11/24-06:03:19.533570 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294362: 922615 keys, 35084404 bytes, temperature: kWarm 2024/11/24-06:03:19.533650 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399399533605, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294362, "file_size": 35084404, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4339385708, "largest_seqno": 4470947111, "table_properties": {"data_size": 29105136, "index_size": 1778368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25833220, "raw_average_key_size": 28, "raw_value_size": 108384578, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922615, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399396, "oldest_key_time": 0, "file_creation_time": 1732399396, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294362, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:19.569832 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294363.sst) /dev/shm/ToplingTemp/Topling-285124-bPjZjU 2024/11/24-06:03:20.435651 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294363.sst): first pass time = 0.87's, 155.025'MB/sec 2024/11/24-06:03:21.769208 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294363.sst): old prealloc_size = 147639500, real_size = 32983968 2024/11/24-06:03:21.769727 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294363.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924690 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.15 avg-zval = 33.52 2024/11/24-06:03:21.770514 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294363.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:22.614823 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294363.sst): second pass time = 1.33's, 81.275'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.846'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3529.307'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 292.174'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35398.088'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.275'MB/sec zip pipeline throughput = 81.275'MB/sec entries = 924690 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 33.52 usrkeys = 924690 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924690 bytes = 3236432 } seq expand size = 6472830 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5147 value = 3.4951 dict = 1.00 all = 3.6201 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 2252.878261 GB avg = 0.200 KB (by entry num) total key len = 307.232963 GB avg = 0.027 KB total ukey len = 237.604727 GB avg = 0.015 KB total ukey num = 15.751161165 Billion total entry num = 11.292475113 Billion write speed all = 33.645653294 MB/sec (with seq num) write speed all = 32.458383932 MB/sec (without seq num) 2024/11/24-06:03:22.662211 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294363: 924690 keys, 37075940 bytes, temperature: kWarm 2024/11/24-06:03:22.662281 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399402662243, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294363, "file_size": 37075940, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4470947211, "table_properties": {"data_size": 30993744, "index_size": 1758848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25891320, "raw_average_key_size": 28, "raw_value_size": 108326528, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924690, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399399, "oldest_key_time": 0, "file_creation_time": 1732399399, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294363, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:22.713553 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294364.sst) /dev/shm/ToplingTemp/Topling-285124-LNNVnV 2024/11/24-06:03:23.582212 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294364.sst): first pass time = 0.87's, 154.518'MB/sec 2024/11/24-06:03:24.910521 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294364.sst): old prealloc_size = 147639500, real_size = 31769008 2024/11/24-06:03:24.911098 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294364.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927574 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0298 GB avg-val = 116.70 avg-zval = 32.10 2024/11/24-06:03:24.911878 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294364.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:25.740631 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294364.sst): second pass time = 1.33's, 81.540'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.212'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3180.134'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 278.493'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33951.288'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.540'MB/sec zip pipeline throughput = 81.540'MB/sec entries = 927574 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.70 avg-zval = 32.10 usrkeys = 927574 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.70 avg-zval = 32.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927574 bytes = 3246528 } seq expand size = 6493018 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0298 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5127 value = 3.6358 dict = 1.00 all = 3.7418 } Zip/UnZip{ index = 0.0951 value = 0.2750 dict = 1.00 all = 0.2673 } ---------------------------- total value len = 2252.992999 GB avg = 0.199 KB (by entry num) total key len = 307.258935 GB avg = 0.027 KB total ukey len = 237.623279 GB avg = 0.015 KB total ukey num = 15.752088739 Billion total entry num = 11.293402687 Billion write speed all = 33.646120361 MB/sec (with seq num) write speed all = 32.458802251 MB/sec (without seq num) 2024/11/24-06:03:25.788607 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294364: 927574 keys, 35869850 bytes, temperature: kWarm 2024/11/24-06:03:25.788676 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399405788641, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294364, "file_size": 35869850, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4470947130, "table_properties": {"data_size": 29772256, "index_size": 1764672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25972072, "raw_average_key_size": 28, "raw_value_size": 108245744, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927574, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399402, "oldest_key_time": 0, "file_creation_time": 1732399402, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294364, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:25.826917 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294365.sst) /dev/shm/ToplingTemp/Topling-285124-jJoXoU 2024/11/24-06:03:26.815073 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294365.sst): first pass time = 0.99's, 135.832'MB/sec 2024/11/24-06:03:28.117775 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294365.sst): old prealloc_size = 147639500, real_size = 31405904 2024/11/24-06:03:28.118378 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294365.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.99 2024/11/24-06:03:28.120395 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294365.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:28.935912 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294365.sst): second pass time = 1.30's, 83.320'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.371'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3095.862'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 275.339'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 26858.013'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.320'MB/sec zip pipeline throughput = 83.320'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.99 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3764 value = 3.6892 dict = 1.00 all = 3.7826 } Zip/UnZip{ index = 0.0964 value = 0.2711 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 2253.107912 GB avg = 0.199 KB (by entry num) total key len = 307.284675 GB avg = 0.027 KB total ukey len = 237.641665 GB avg = 0.015 KB total ukey num = 15.753008039 Billion total entry num = 11.294321987 Billion write speed all = 33.646555916 MB/sec (with seq num) write speed all = 32.459191015 MB/sec (without seq num) 2024/11/24-06:03:28.982100 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294365: 919300 keys, 35482674 bytes, temperature: kWarm 2024/11/24-06:03:28.982171 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399408982134, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294365, "file_size": 35482674, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4280321373, "largest_seqno": 4432682123, "table_properties": {"data_size": 29403968, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399405, "oldest_key_time": 0, "file_creation_time": 1732399405, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294365, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:29.021081 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294366.sst) /dev/shm/ToplingTemp/Topling-285124-dmtjGW 2024/11/24-06:03:30.162004 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294366.sst): first pass time = 1.14's, 117.643'MB/sec 2024/11/24-06:03:31.521493 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294366.sst): old prealloc_size = 147639500, real_size = 33158864 2024/11/24-06:03:31.522066 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294366.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927720 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.67 avg-zval = 33.59 2024/11/24-06:03:31.523074 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294366.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:32.384724 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294366.sst): second pass time = 1.36's, 79.701'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.165'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3212.800'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 289.345'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35540.218'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.701'MB/sec zip pipeline throughput = 79.701'MB/sec entries = 927720 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.67 avg-zval = 33.59 usrkeys = 927720 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.67 avg-zval = 33.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927720 bytes = 3247040 } seq expand size = 6494040 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5144 value = 3.4735 dict = 1.00 all = 3.6022 } Zip/UnZip{ index = 0.0951 value = 0.2879 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 2253.222647 GB avg = 0.199 KB (by entry num) total key len = 307.310651 GB avg = 0.027 KB total ukey len = 237.660219 GB avg = 0.015 KB total ukey num = 15.753935759 Billion total entry num = 11.295249707 Billion write speed all = 33.646880108 MB/sec (with seq num) write speed all = 32.459471492 MB/sec (without seq num) 2024/11/24-06:03:32.432170 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294366: 927720 keys, 37260108 bytes, temperature: kWarm 2024/11/24-06:03:32.432235 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399412432201, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294366, "file_size": 37260108, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4470947614, "table_properties": {"data_size": 31162048, "index_size": 1764672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25976160, "raw_average_key_size": 28, "raw_value_size": 108241572, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927720, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399409, "oldest_key_time": 0, "file_creation_time": 1732399409, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294366, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:32.483127 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294367.sst) /dev/shm/ToplingTemp/Topling-285124-6uy1UV 2024/11/24-06:03:33.505667 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294367.sst): first pass time = 1.02's, 131.264'MB/sec 2024/11/24-06:03:34.824509 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294367.sst): old prealloc_size = 147639500, real_size = 31368704 2024/11/24-06:03:34.825068 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294367.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921060 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0294 GB avg-val = 117.72 avg-zval = 31.89 2024/11/24-06:03:34.827038 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294367.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:35.642991 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294367.sst): second pass time = 1.32's, 82.261'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.314'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3289.478'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 275.471'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35049.997'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.261'MB/sec zip pipeline throughput = 82.261'MB/sec entries = 921060 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.72 avg-zval = 31.89 usrkeys = 921060 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.72 avg-zval = 31.89 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921060 bytes = 3223728 } seq expand size = 6447420 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4461 value = 3.6912 dict = 1.00 all = 3.7861 } Zip/UnZip{ index = 0.0957 value = 0.2709 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 2253.337523 GB avg = 0.199 KB (by entry num) total key len = 307.336441 GB avg = 0.027 KB total ukey len = 237.678640 GB avg = 0.015 KB total ukey num = 15.754856819 Billion total entry num = 11.296170767 Billion write speed all = 33.647287912 MB/sec (with seq num) write speed all = 32.459833310 MB/sec (without seq num) 2024/11/24-06:03:35.687163 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294367: 921060 keys, 35449912 bytes, temperature: kWarm 2024/11/24-06:03:35.687232 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399415687196, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294367, "file_size": 35449912, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4330553353, "largest_seqno": 4470947167, "table_properties": {"data_size": 29374768, "index_size": 1763456, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25789680, "raw_average_key_size": 28, "raw_value_size": 108428060, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921060, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399412, "oldest_key_time": 0, "file_creation_time": 1732399412, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294367, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:35.728750 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294368.sst) /dev/shm/ToplingTemp/Topling-285124-iaomTW 2024/11/24-06:03:36.574308 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294368.sst): first pass time = 0.85's, 158.741'MB/sec 2024/11/24-06:03:37.943198 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294368.sst): old prealloc_size = 147639500, real_size = 32974896 2024/11/24-06:03:37.943745 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294368.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925650 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.00 avg-zval = 33.47 2024/11/24-06:03:37.944620 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294368.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:38.796909 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294368.sst): second pass time = 1.37's, 79.186'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.423'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3372.512'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 285.876'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 23372.815'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.186'MB/sec zip pipeline throughput = 79.186'MB/sec entries = 925650 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.00 avg-zval = 33.47 usrkeys = 925650 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.00 avg-zval = 33.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925650 bytes = 3239792 } seq expand size = 6479550 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5145 value = 3.4955 dict = 1.00 all = 3.6207 } Zip/UnZip{ index = 0.0951 value = 0.2861 dict = 1.00 all = 0.2762 } ---------------------------- total value len = 2253.452302 GB avg = 0.199 KB (by entry num) total key len = 307.362359 GB avg = 0.027 KB total ukey len = 237.697153 GB avg = 0.015 KB total ukey num = 15.755782469 Billion total entry num = 11.297096417 Billion write speed all = 33.647742220 MB/sec (with seq num) write speed all = 32.460239527 MB/sec (without seq num) 2024/11/24-06:03:38.844333 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294368: 925650 keys, 37069334 bytes, temperature: kWarm 2024/11/24-06:03:38.844409 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399418844374, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294368, "file_size": 37069334, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4470947920, "table_properties": {"data_size": 30982624, "index_size": 1760704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25918200, "raw_average_key_size": 28, "raw_value_size": 108299572, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925650, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399415, "oldest_key_time": 0, "file_creation_time": 1732399415, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294368, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:38.883573 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294369.sst) /dev/shm/ToplingTemp/Topling-285124-lOmywY 2024/11/24-06:03:39.893826 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294369.sst): first pass time = 1.01's, 132.861'MB/sec 2024/11/24-06:03:41.238171 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294369.sst): old prealloc_size = 147639500, real_size = 31612048 2024/11/24-06:03:41.238714 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294369.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927816 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0296 GB avg-val = 116.66 avg-zval = 31.91 2024/11/24-06:03:41.240709 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294369.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:42.069282 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294369.sst): second pass time = 1.34's, 80.560'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.978'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.117'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 272.801'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27418.475'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.560'MB/sec zip pipeline throughput = 80.560'MB/sec entries = 927816 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.66 avg-zval = 31.91 usrkeys = 927816 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.66 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927816 bytes = 3247376 } seq expand size = 6494712 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4442 value = 3.6563 dict = 1.00 all = 3.7578 } Zip/UnZip{ index = 0.0957 value = 0.2735 dict = 1.00 all = 0.2661 } ---------------------------- total value len = 2253.567036 GB avg = 0.199 KB (by entry num) total key len = 307.388338 GB avg = 0.027 KB total ukey len = 237.715710 GB avg = 0.015 KB total ukey num = 15.756710285 Billion total entry num = 11.298024233 Billion write speed all = 33.648144341 MB/sec (with seq num) write speed all = 32.460595181 MB/sec (without seq num) 2024/11/24-06:03:42.115381 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294369: 927816 keys, 35717104 bytes, temperature: kWarm 2024/11/24-06:03:42.115453 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399422115418, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294369, "file_size": 35717104, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4470947588, "table_properties": {"data_size": 29603200, "index_size": 1776704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25978848, "raw_average_key_size": 28, "raw_value_size": 108239000, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927816, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399418, "oldest_key_time": 0, "file_creation_time": 1732399418, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294369, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:42.159515 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294370.sst) /dev/shm/ToplingTemp/Topling-285124-KvB61W 2024/11/24-06:03:42.952098 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294370.sst): first pass time = 0.79's, 169.351'MB/sec 2024/11/24-06:03:44.286033 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294370.sst): old prealloc_size = 147639500, real_size = 30915392 2024/11/24-06:03:44.286600 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294370.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926000 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 116.94 avg-zval = 31.22 2024/11/24-06:03:44.287382 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294370.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:45.106902 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294370.sst): second pass time = 1.33's, 81.259'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.651'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3266.415'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 269.235'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31653.692'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.259'MB/sec zip pipeline throughput = 81.259'MB/sec entries = 926000 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.94 avg-zval = 31.22 usrkeys = 926000 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.94 avg-zval = 31.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926000 bytes = 3241008 } seq expand size = 6482000 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0350 GB } UnZip/Zip{ index = 10.4452 value = 3.7457 dict = 1.00 all = 3.8337 } Zip/UnZip{ index = 0.0957 value = 0.2670 dict = 1.00 all = 0.2608 } ---------------------------- total value len = 2253.681808 GB avg = 0.199 KB (by entry num) total key len = 307.414266 GB avg = 0.027 KB total ukey len = 237.734230 GB avg = 0.015 KB total ukey num = 15.757636285 Billion total entry num = 11.298950233 Billion write speed all = 33.648650022 MB/sec (with seq num) write speed all = 32.461050928 MB/sec (without seq num) 2024/11/24-06:03:45.152341 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294370: 926000 keys, 35009600 bytes, temperature: kWarm 2024/11/24-06:03:45.152417 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399425152382, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294370, "file_size": 35009600, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4470946876, "table_properties": {"data_size": 28910640, "index_size": 1773056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928000, "raw_average_key_size": 28, "raw_value_size": 108289848, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926000, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399422, "oldest_key_time": 0, "file_creation_time": 1732399422, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294370, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:45.192892 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294371.sst) /dev/shm/ToplingTemp/Topling-285124-wiSjOV 2024/11/24-06:03:46.013990 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294371.sst): first pass time = 0.82's, 163.472'MB/sec 2024/11/24-06:03:47.278439 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294371.sst): old prealloc_size = 147639500, real_size = 27855552 2024/11/24-06:03:47.279000 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294371.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0258 GB avg-val = 114.00 avg-zval = 27.31 2024/11/24-06:03:47.279770 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294371.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:03:48.121700 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294371.sst): second pass time = 1.26's, 85.268'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 37.815'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3364.457'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.333'MB/sec write SST data time = 0.13's, 240.487'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 35670.222'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.268'MB/sec zip pipeline throughput = 85.268'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.31 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0258 GB dict = 1.07 MB all = 0.0319 GB } UnZip/Zip{ index = 10.4450 value = 4.1750 dict = 1.00 all = 4.2092 } Zip/UnZip{ index = 0.0957 value = 0.2395 dict = 1.00 all = 0.2376 } ---------------------------- total value len = 2253.796176 GB avg = 0.199 KB (by entry num) total key len = 307.440731 GB avg = 0.027 KB total ukey len = 237.753134 GB avg = 0.015 KB total ukey num = 15.758581481 Billion total entry num = 11.299895429 Billion write speed all = 33.649167528 MB/sec (with seq num) write speed all = 32.461516130 MB/sec (without seq num) 2024/11/24-06:03:48.163895 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294371: 945196 keys, 31886768 bytes, temperature: kWarm 2024/11/24-06:03:48.163966 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399428163930, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294371, "file_size": 31886768, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4290937813, "largest_seqno": 4392174876, "table_properties": {"data_size": 25809200, "index_size": 1809856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399425, "oldest_key_time": 0, "file_creation_time": 1732399425, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294371, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:48.202538 139957676762688 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294372.sst) /dev/shm/ToplingTemp/Topling-285124-QlwXpX 2024/11/24-06:03:48.679570 139957676762688 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294372.sst): first pass time = 0.48's, 247.069'MB/sec 2024/11/24-06:03:49.849577 139957676762688 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294372.sst): old prealloc_size = 147639500, real_size = 26819008 2024/11/24-06:03:49.850108 139957676762688 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294372.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 826758 prefix = 4 raw-key = 0.0132 GB zip-key = 0.0016 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0947 GB zip-val = 0.0250 GB avg-val = 114.55 avg-zval = 30.29 2024/11/24-06:03:49.850850 139957676762688 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294372.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0021 GB, waited 0.000 sec, Key+Value = 0.118 GB 2024/11/24-06:03:50.626742 139957676762688 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00/294372.sst): second pass time = 1.17's, 80.993'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 39.191'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3080.215'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.315'MB/sec write SST data time = 0.12's, 251.287'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.946'MB, throughput = 30246.826'MB/sec zip my value time = 1.17's, unzip length = 0.095'GB zip my value throughput = 80.993'MB/sec zip pipeline throughput = 80.993'MB/sec entries = 826758 avg-key = 28.00 avg-zkey = 1.90 avg-val = 114.55 avg-zval = 30.29 usrkeys = 826758 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.55 avg-zval = 30.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 826758 bytes = 2893664 } seq expand size = 5787306 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0165 GB value = 0.0947 GB dict = 0.95 MB all = 0.1179 GB } __ZipSize{ index = 0.0016 GB value = 0.0250 GB dict = 0.95 MB all = 0.0305 GB } UnZip/Zip{ index = 10.5119 value = 3.7821 dict = 1.00 all = 3.8694 } Zip/UnZip{ index = 0.0951 value = 0.2644 dict = 1.00 all = 0.2584 } ---------------------------- total value len = 2253.896665 GB avg = 0.199 KB (by entry num) total key len = 307.463881 GB avg = 0.027 KB total ukey len = 237.769669 GB avg = 0.015 KB total ukey num = 15.759408239 Billion total entry num = 11.300722187 Billion write speed all = 33.649684423 MB/sec (with seq num) write speed all = 32.461985219 MB/sec (without seq num) 2024/11/24-06:03:50.668932 139957676762688 [compaction_job.cc:1948] [default] [JOB 42554] Generated table #294372: 826758 keys, 30457100 bytes, temperature: kWarm 2024/11/24-06:03:50.669029 139957676762688 EVENT_LOG_v1 {"time_micros": 1732399430668967, "cf_name": "default", "job": 42554, "event": "table_file_creation", "file_number": 294372, "file_size": 30457100, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4470662387, "table_properties": {"data_size": 25039152, "index_size": 1572992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 23149224, "raw_average_key_size": 28, "raw_value_size": 94701336, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 826758, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732399428, "oldest_key_time": 0, "file_creation_time": 1732399428, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294372, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:03:50.694062 139957676762688 [compaction_job.cc:703] job-42554: subcompact[0], size: 0.803756 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42554/att-00 [294350,294351,294352,294353,294354,294355,294356,294357,294358,294359,294360,294361,294362,294363,294364,294365,294366,294367,294368,294369,294370,294371,294372] 2024/11/24-06:03:50.825540 139957676762688 (Original Log Time 2024/11/24-06:02:36.687759) EVENT_LOG_v1 {"time_micros": 1732399356687719, "job": 42554, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [294346], "files_L3": [294271, 294272, 294273, 294274, 294275, 294312, 294313, 294314, 294315, 294316, 294317, 294318, 294319, 294320, 294321, 294322, 294323, 294284, 294285, 294286, 294287, 294288, 294289], "score": 0.941382, "input_data_size": 827692149, "oldest_snapshot_seqno": 4470960830} 2024/11/24-06:03:50.825544 139957676762688 (Original Log Time 2024/11/24-06:03:50.823912) [compaction_job.cc:2022] [default] [JOB 42554] Compacted 1@2 + 23@3 files to L3 => 803755679 bytes 2024/11/24-06:03:50.825545 139957676762688 (Original Log Time 2024/11/24-06:03:50.825468) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 11.2 rd, 10.9 wr, level 3, files in(1, 23) out(23 +0 blob) MB in(27.1, 762.3 +0.0 blob) out(766.5 +0.0 blob), read-write-amplify(57.4) write-amplify(28.3) OK, records in: 20848852, records dropped: 101776 output_compression: Snappy 2024/11/24-06:03:50.825549 139957676762688 (Original Log Time 2024/11/24-06:03:50.825513) EVENT_LOG_v1 {"time_micros": 1732399430825492, "cf": "default", "job": 42554, "event": "compaction_finished", "compaction_time_micros": 74006312, "compaction_time_cpu_micros": 31535522, "output_level": 3, "num_output_files": 23, "total_output_size": 803755679, "num_input_records": 20848852, "num_output_records": 20747076, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 548927688, "file_range_sync_nanos": 0, "file_fsync_nanos": 16459, "file_prepare_write_nanos": 60945, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/24-06:03:50.952973 139957676762688 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732399356,"executesMs":74274,"compactionJobId":42554,"attempt":0,"compactionInputRawBytes":3021232492,"compactionInputZipBytes":827692149,"compactionOutputRawBytes":0,"compactionOutputZipBytes":803755679,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-48c1e0232126a8c6791d47cff407cd40-ae976320d57a5d10-00"}}, response = 2024/11/24-06:03:50.953107 139957676762688 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42554/att-00: olev 3, work 74.275 s, result 7.743 ms, install 1.652 ms, input{raw 2.814 GiB zip 789.349 MiB}