2024/11/24-05:04:41.079738 139957189985856 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00: fork to child time = 0.007969 sec 2024/11/24-05:04:41.080495 139957189985856 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00 2024/11/24-05:04:41.086076 139957189985856 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00 2024/11/24-05:04:41.086953 139957189985856 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-05:04:41.087201 139957189985856 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-05:04:41.138537 139957189985856 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41943/att-00/MANIFEST-206347 2024/11/24-05:04:41.163609 139957189985856 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41943/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 292922, last_sequence is 4451947033, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 292735 2024/11/24-05:04:41.163615 139957189985856 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-05:04:41.163732 139957189985856 [version_set.cc:5721] Creating manifest 292922 2024/11/24-05:04:41.201498 139957189985856 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-05:04:41.225114 139957189985856 [compaction_job.cc:2437] [default] [JOB 41943] Compacting 1@2 + 21@3 files to L3, score 0.98, subcompactions 1 : 1 2024/11/24-05:04:41.225127 139957189985856 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 31442 Base level 2, inputs: [292870(27MB)], [292920(19MB) 292836(33MB) 292837(30MB) 292838(33MB) 292839(32MB) 292840(35MB) 292841(33MB) 292842(23MB) 292791(33MB) 292792(33MB) 292793(33MB) 292794(32MB) 292795(34MB) 292796(33MB) 292797(35MB) 292798(33MB) 292799(35MB) 292800(33MB) 292801(35MB) 292802(33MB) 292803(21MB)] 2024/11/24-05:04:41.231727 139957189985856 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:04:41.231750 139957189985856 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:04:41.231764 139957189985856 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-05:04:41.231769 139957189985856 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:04:41.231778 139957189985856 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-05:04:41.636457 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292923.sst) /dev/shm/ToplingTemp/Topling-276011-cVm5XP 2024/11/24-05:04:42.737029 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292923.sst): first pass time = 1.10's, 121.958'MB/sec 2024/11/24-05:04:44.357833 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292923.sst): old prealloc_size = 147639500, real_size = 32134848 2024/11/24-05:04:44.358593 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292923.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920824 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0302 GB avg-val = 117.76 avg-zval = 32.75 2024/11/24-05:04:44.359504 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292923.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:04:45.440262 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292923.sst): second pass time = 1.62's, 66.983'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.08's, 33.459'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2379.973'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.250'MB/sec write SST data time = 0.16's, 227.298'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35331.617'MB/sec zip my value time = 1.62's, unzip length = 0.108'GB zip my value throughput = 66.983'MB/sec zip pipeline throughput = 66.983'MB/sec entries = 920824 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.76 avg-zval = 32.75 usrkeys = 920824 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.76 avg-zval = 32.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920824 bytes = 3222896 } seq expand size = 6445768 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0362 GB } UnZip/Zip{ index = 10.5121 value = 3.5962 dict = 1.00 all = 3.7059 } Zip/UnZip{ index = 0.0951 value = 0.2781 dict = 1.00 all = 0.2698 } ---------------------------- total value len = 2251.440454 GB avg = 0.200 KB (by entry num) total key len = 306.908746 GB avg = 0.027 KB total ukey len = 237.373144 GB avg = 0.015 KB total ukey num = 15.739581987 Billion total entry num = 11.280895935 Billion write speed all = 35.251970379 MB/sec (with seq num) write speed all = 34.008437840 MB/sec (without seq num) 2024/11/24-05:04:45.486811 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292923: 920824 keys, 36217408 bytes, temperature: kWarm 2024/11/24-05:04:45.486898 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395885486844, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292923, "file_size": 36217408, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4451516755, "table_properties": {"data_size": 30152496, "index_size": 1751936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25783072, "raw_average_key_size": 28, "raw_value_size": 108434796, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920824, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395881, "oldest_key_time": 0, "file_creation_time": 1732395881, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292923, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:04:45.524411 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292924.sst) /dev/shm/ToplingTemp/Topling-276011-vBTl0O 2024/11/24-05:04:46.474783 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292924.sst): first pass time = 0.95's, 141.232'MB/sec 2024/11/24-05:04:47.912885 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292924.sst): old prealloc_size = 147639500, real_size = 29566400 2024/11/24-05:04:47.913469 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292924.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936362 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1080 GB zip-val = 0.0275 GB avg-val = 115.34 avg-zval = 29.41 2024/11/24-05:04:47.914249 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292924.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:04:48.784116 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292924.sst): second pass time = 1.44's, 75.173'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 38.537'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3204.391'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.14's, 242.320'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 36873.759'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.173'MB/sec zip pipeline throughput = 75.173'MB/sec entries = 936362 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.34 avg-zval = 29.41 usrkeys = 936362 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.34 avg-zval = 29.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936362 bytes = 3160240 } seq expand size = 6554534 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0275 GB dict = 1.08 MB all = 0.0336 GB } UnZip/Zip{ index = 10.4426 value = 3.9217 dict = 1.00 all = 3.9977 } Zip/UnZip{ index = 0.0958 value = 0.2550 dict = 1.00 all = 0.2501 } ---------------------------- total value len = 2251.555008 GB avg = 0.200 KB (by entry num) total key len = 306.934964 GB avg = 0.027 KB total ukey len = 237.391871 GB avg = 0.015 KB total ukey num = 15.740518349 Billion total entry num = 11.281832297 Billion write speed all = 35.252285837 MB/sec (with seq num) write speed all = 34.008707378 MB/sec (without seq num) 2024/11/24-05:04:48.827775 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292924: 936362 keys, 33573758 bytes, temperature: kWarm 2024/11/24-05:04:48.827847 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395888827811, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292924, "file_size": 33573758, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4331901030, "largest_seqno": 4451516919, "table_properties": {"data_size": 27538800, "index_size": 1793344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26218136, "raw_average_key_size": 28, "raw_value_size": 107999636, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936362, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395885, "oldest_key_time": 0, "file_creation_time": 1732395885, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292924, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:04:48.869736 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292925.sst) /dev/shm/ToplingTemp/Topling-276011-tC1MdS 2024/11/24-05:04:49.705082 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292925.sst): first pass time = 0.84's, 160.682'MB/sec 2024/11/24-05:04:51.178674 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292925.sst): old prealloc_size = 147639500, real_size = 32751600 2024/11/24-05:04:51.179259 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292925.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932370 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0307 GB avg-val = 115.95 avg-zval = 32.96 2024/11/24-05:04:51.180260 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292925.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:04:52.056467 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292925.sst): second pass time = 1.47's, 73.406'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.999'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3183.180'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 275.484'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34344.549'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.406'MB/sec zip pipeline throughput = 85.156'MB/sec entries = 932370 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.95 avg-zval = 32.96 usrkeys = 932370 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.95 avg-zval = 32.96 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932370 bytes = 3263312 } seq expand size = 6526590 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.4451 value = 3.5178 dict = 1.00 all = 3.6407 } Zip/UnZip{ index = 0.0957 value = 0.2843 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 2251.669646 GB avg = 0.200 KB (by entry num) total key len = 306.961070 GB avg = 0.027 KB total ukey len = 237.410518 GB avg = 0.015 KB total ukey num = 15.741450719 Billion total entry num = 11.282764667 Billion write speed all = 35.252635611 MB/sec (with seq num) write speed all = 34.009010451 MB/sec (without seq num) 2024/11/24-05:04:52.101533 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292925: 932370 keys, 36865602 bytes, temperature: kWarm 2024/11/24-05:04:52.101605 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395892101566, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292925, "file_size": 36865602, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4451514296, "table_properties": {"data_size": 30733024, "index_size": 1785280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26106360, "raw_average_key_size": 28, "raw_value_size": 108111508, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932370, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395888, "oldest_key_time": 0, "file_creation_time": 1732395888, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292925, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:04:52.138838 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292926.sst) /dev/shm/ToplingTemp/Topling-276011-C3jIPS 2024/11/24-05:04:53.141747 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292926.sst): first pass time = 1.00's, 133.833'MB/sec 2024/11/24-05:04:54.580561 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292926.sst): old prealloc_size = 147639500, real_size = 31357264 2024/11/24-05:04:54.581129 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292926.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.95 2024/11/24-05:04:54.581870 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292926.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:04:55.410197 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292926.sst): second pass time = 1.44's, 75.469'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.573'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3236.248'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 269.298'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32335.270'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.469'MB/sec zip pipeline throughput = 76.042'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.95 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6938 dict = 1.00 all = 3.8001 } Zip/UnZip{ index = 0.0957 value = 0.2707 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 2251.784559 GB avg = 0.200 KB (by entry num) total key len = 306.986810 GB avg = 0.027 KB total ukey len = 237.428904 GB avg = 0.015 KB total ukey num = 15.742370019 Billion total entry num = 11.283683967 Billion write speed all = 35.252944567 MB/sec (with seq num) write speed all = 34.009275546 MB/sec (without seq num) 2024/11/24-05:04:55.453613 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292926: 919300 keys, 35319090 bytes, temperature: kWarm 2024/11/24-05:04:55.453683 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395895453648, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292926, "file_size": 35319090, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4321983242, "largest_seqno": 4440984996, "table_properties": {"data_size": 29367040, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395892, "oldest_key_time": 0, "file_creation_time": 1732395892, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292926, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:04:55.490009 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292927.sst) /dev/shm/ToplingTemp/Topling-276011-0cWeuQ 2024/11/24-05:04:56.486495 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292927.sst): first pass time = 1.00's, 134.696'MB/sec 2024/11/24-05:04:57.959295 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292927.sst): old prealloc_size = 147639500, real_size = 31572912 2024/11/24-05:04:57.959836 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292927.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 936243 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0296 GB avg-val = 115.36 avg-zval = 31.57 2024/11/24-05:04:57.960638 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292927.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:04:58.830936 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292927.sst): second pass time = 1.47's, 73.396'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 40.957'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3460.118'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.14's, 260.011'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 36056.114'MB/sec zip my value time = 1.47's, unzip length = 0.108'GB zip my value throughput = 73.396'MB/sec zip pipeline throughput = 86.053'MB/sec entries = 936243 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.36 avg-zval = 31.57 usrkeys = 936243 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.36 avg-zval = 31.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 936243 bytes = 3276864 } seq expand size = 6553701 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5107 value = 3.6540 dict = 1.00 all = 3.7598 } Zip/UnZip{ index = 0.0951 value = 0.2737 dict = 1.00 all = 0.2660 } ---------------------------- total value len = 2251.899115 GB avg = 0.200 KB (by entry num) total key len = 307.013025 GB avg = 0.027 KB total ukey len = 237.447629 GB avg = 0.015 KB total ukey num = 15.743306262 Billion total entry num = 11.284620210 Billion write speed all = 35.253222588 MB/sec (with seq num) write speed all = 34.009508990 MB/sec (without seq num) 2024/11/24-05:04:58.879901 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292927: 936243 keys, 35698572 bytes, temperature: kWarm 2024/11/24-05:04:58.879973 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395898879938, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292927, "file_size": 35698572, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4451516936, "table_properties": {"data_size": 29557152, "index_size": 1781504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26214804, "raw_average_key_size": 28, "raw_value_size": 108003062, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 936243, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395895, "oldest_key_time": 0, "file_creation_time": 1732395895, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292927, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:04:58.917667 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292928.sst) /dev/shm/ToplingTemp/Topling-276011-naLglR 2024/11/24-05:04:59.907768 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292928.sst): first pass time = 0.99's, 135.565'MB/sec 2024/11/24-05:05:01.350435 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292928.sst): old prealloc_size = 147639500, real_size = 31524736 2024/11/24-05:05:01.350963 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292928.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.13 2024/11/24-05:05:01.351760 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292928.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:02.194216 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292928.sst): second pass time = 1.44's, 75.239'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.058'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3385.347'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.14's, 260.797'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33425.033'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.239'MB/sec zip pipeline throughput = 75.239'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.13 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4443 value = 3.6729 dict = 1.00 all = 3.7822 } Zip/UnZip{ index = 0.0957 value = 0.2723 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 2252.014028 GB avg = 0.200 KB (by entry num) total key len = 307.038766 GB avg = 0.027 KB total ukey len = 237.466015 GB avg = 0.015 KB total ukey num = 15.744225562 Billion total entry num = 11.285539510 Billion write speed all = 35.253526847 MB/sec (with seq num) write speed all = 34.009769560 MB/sec (without seq num) 2024/11/24-05:05:02.238431 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292928: 919300 keys, 35486578 bytes, temperature: kWarm 2024/11/24-05:05:02.238509 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395902238469, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292928, "file_size": 35486578, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4292961509, "largest_seqno": 4403909920, "table_properties": {"data_size": 29534320, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395898, "oldest_key_time": 0, "file_creation_time": 1732395898, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292928, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:02.276168 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292929.sst) /dev/shm/ToplingTemp/Topling-276011-4GLoNP 2024/11/24-05:05:03.163315 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292929.sst): first pass time = 0.89's, 151.297'MB/sec 2024/11/24-05:05:04.603267 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292929.sst): old prealloc_size = 147639500, real_size = 30790736 2024/11/24-05:05:04.603795 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292929.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932513 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0288 GB avg-val = 115.93 avg-zval = 30.87 2024/11/24-05:05:04.604579 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292929.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:05.456644 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292929.sst): second pass time = 1.44's, 75.120'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 40.903'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3509.536'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.14's, 249.954'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35735.654'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.120'MB/sec zip pipeline throughput = 75.120'MB/sec entries = 932513 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.93 avg-zval = 30.87 usrkeys = 932513 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.93 avg-zval = 30.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932513 bytes = 3263808 } seq expand size = 6527591 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0288 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.5138 value = 3.7559 dict = 1.00 all = 3.8452 } Zip/UnZip{ index = 0.0951 value = 0.2662 dict = 1.00 all = 0.2601 } ---------------------------- total value len = 2252.128663 GB avg = 0.200 KB (by entry num) total key len = 307.064876 GB avg = 0.027 KB total ukey len = 237.484666 GB avg = 0.015 KB total ukey num = 15.745158075 Billion total entry num = 11.286472023 Billion write speed all = 35.253881336 MB/sec (with seq num) write speed all = 34.010077179 MB/sec (without seq num) 2024/11/24-05:05:05.501460 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292929: 932513 keys, 34905246 bytes, temperature: kWarm 2024/11/24-05:05:05.501557 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395905501504, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292929, "file_size": 34905246, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4451516946, "table_properties": {"data_size": 28783552, "index_size": 1773888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26110364, "raw_average_key_size": 28, "raw_value_size": 108107446, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932513, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395902, "oldest_key_time": 0, "file_creation_time": 1732395902, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292929, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:05.538664 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292930.sst) /dev/shm/ToplingTemp/Topling-276011-PTlcaR 2024/11/24-05:05:06.437847 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292930.sst): first pass time = 0.90's, 149.273'MB/sec 2024/11/24-05:05:07.852877 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292930.sst): old prealloc_size = 147639500, real_size = 31378576 2024/11/24-05:05:07.853426 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292930.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.97 2024/11/24-05:05:07.854174 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292930.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:08.690544 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292930.sst): second pass time = 1.41's, 76.728'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.191'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3346.347'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.327'MB/sec write SST data time = 0.13's, 262.211'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36588.993'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.728'MB/sec zip pipeline throughput = 76.728'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.97 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6912 dict = 1.00 all = 3.7979 } Zip/UnZip{ index = 0.0957 value = 0.2709 dict = 1.00 all = 0.2633 } ---------------------------- total value len = 2252.243575 GB avg = 0.200 KB (by entry num) total key len = 307.090616 GB avg = 0.027 KB total ukey len = 237.503052 GB avg = 0.015 KB total ukey num = 15.746077375 Billion total entry num = 11.287391323 Billion write speed all = 35.254248370 MB/sec (with seq num) write speed all = 34.010398314 MB/sec (without seq num) 2024/11/24-05:05:08.735624 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292930: 919300 keys, 35340402 bytes, temperature: kWarm 2024/11/24-05:05:08.735694 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395908735658, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292930, "file_size": 35340402, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4272732412, "largest_seqno": 4381115002, "table_properties": {"data_size": 29388352, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395905, "oldest_key_time": 0, "file_creation_time": 1732395905, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292930, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:08.774614 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292931.sst) /dev/shm/ToplingTemp/Topling-276011-tQ4uSP 2024/11/24-05:05:09.759916 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292931.sst): first pass time = 0.99's, 136.226'MB/sec 2024/11/24-05:05:11.196462 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292931.sst): old prealloc_size = 147639500, real_size = 30453632 2024/11/24-05:05:11.196999 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292931.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928992 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.48 avg-zval = 30.62 2024/11/24-05:05:11.197804 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292931.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:12.020221 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292931.sst): second pass time = 1.44's, 75.366'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.810'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3462.877'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 270.725'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 39392.323'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.366'MB/sec zip pipeline throughput = 75.366'MB/sec entries = 928992 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.48 avg-zval = 30.62 usrkeys = 928992 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.48 avg-zval = 30.62 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928992 bytes = 3135360 } seq expand size = 6502944 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.4447 value = 3.8044 dict = 1.00 all = 3.8971 } Zip/UnZip{ index = 0.0957 value = 0.2629 dict = 1.00 all = 0.2566 } ---------------------------- total value len = 2252.358284 GB avg = 0.200 KB (by entry num) total key len = 307.116628 GB avg = 0.027 KB total ukey len = 237.521631 GB avg = 0.015 KB total ukey num = 15.747006367 Billion total entry num = 11.288320315 Billion write speed all = 35.254569794 MB/sec (with seq num) write speed all = 34.010674417 MB/sec (without seq num) 2024/11/24-05:05:12.066984 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292931: 928992 keys, 34440370 bytes, temperature: kWarm 2024/11/24-05:05:12.067052 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395912067018, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292931, "file_size": 34440370, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4381115003, "largest_seqno": 4451516961, "table_properties": {"data_size": 28442352, "index_size": 1778880, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26011776, "raw_average_key_size": 28, "raw_value_size": 108206092, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928992, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395908, "oldest_key_time": 0, "file_creation_time": 1732395908, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292931, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:12.107287 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292932.sst) /dev/shm/ToplingTemp/Topling-276011-oulziT 2024/11/24-05:05:13.052784 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292932.sst): first pass time = 0.95's, 141.961'MB/sec 2024/11/24-05:05:14.507304 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292932.sst): old prealloc_size = 147639500, real_size = 32286864 2024/11/24-05:05:14.507816 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292932.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921806 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0303 GB avg-val = 117.60 avg-zval = 32.86 2024/11/24-05:05:14.508589 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292932.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:15.328677 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292932.sst): second pass time = 1.45's, 74.576'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 44.283'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3624.806'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.331'MB/sec write SST data time = 0.13's, 289.463'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33739.055'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.576'MB/sec zip pipeline throughput = 93.684'MB/sec entries = 921806 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.60 avg-zval = 32.86 usrkeys = 921806 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.60 avg-zval = 32.86 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921806 bytes = 3226336 } seq expand size = 6452642 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.4451 value = 3.5788 dict = 1.00 all = 3.6901 } Zip/UnZip{ index = 0.0957 value = 0.2794 dict = 1.00 all = 0.2710 } ---------------------------- total value len = 2252.473144 GB avg = 0.200 KB (by entry num) total key len = 307.142439 GB avg = 0.027 KB total ukey len = 237.540068 GB avg = 0.015 KB total ukey num = 15.747928173 Billion total entry num = 11.289242121 Billion write speed all = 35.254900803 MB/sec (with seq num) write speed all = 34.010960537 MB/sec (without seq num) 2024/11/24-05:05:15.376741 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292932: 921806 keys, 36372152 bytes, temperature: kWarm 2024/11/24-05:05:15.376836 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395915376776, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292932, "file_size": 36372152, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4451516903, "table_properties": {"data_size": 30291200, "index_size": 1765056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25810568, "raw_average_key_size": 28, "raw_value_size": 108407260, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921806, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395912, "oldest_key_time": 0, "file_creation_time": 1732395912, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292932, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:15.416411 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292933.sst) /dev/shm/ToplingTemp/Topling-276011-fRpqnR 2024/11/24-05:05:16.245217 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292933.sst): first pass time = 0.83's, 161.949'MB/sec 2024/11/24-05:05:17.785561 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292933.sst): old prealloc_size = 147639500, real_size = 30924048 2024/11/24-05:05:17.786099 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292933.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924716 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.14 avg-zval = 31.28 2024/11/24-05:05:17.786865 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292933.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:18.604863 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292933.sst): second pass time = 1.54's, 70.376'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.598'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3422.814'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 269.095'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28792.055'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.376'MB/sec zip pipeline throughput = 120.901'MB/sec entries = 924716 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.14 avg-zval = 31.28 usrkeys = 924716 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.14 avg-zval = 31.28 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924716 bytes = 3120928 } seq expand size = 6473012 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4455 value = 3.7454 dict = 1.00 all = 3.8459 } Zip/UnZip{ index = 0.0957 value = 0.2670 dict = 1.00 all = 0.2600 } ---------------------------- total value len = 2252.587943 GB avg = 0.200 KB (by entry num) total key len = 307.168331 GB avg = 0.027 KB total ukey len = 237.558562 GB avg = 0.015 KB total ukey num = 15.748852889 Billion total entry num = 11.290166837 Billion write speed all = 35.255247734 MB/sec (with seq num) write speed all = 34.011261709 MB/sec (without seq num) 2024/11/24-05:05:18.648873 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292933: 924716 keys, 34899116 bytes, temperature: kWarm 2024/11/24-05:05:18.648932 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395918648900, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292933, "file_size": 34899116, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4362818563, "largest_seqno": 4451516979, "table_properties": {"data_size": 28922112, "index_size": 1770560, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25892048, "raw_average_key_size": 28, "raw_value_size": 108325724, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924716, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395915, "oldest_key_time": 0, "file_creation_time": 1732395915, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292933, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:18.689650 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292934.sst) /dev/shm/ToplingTemp/Topling-276011-ZYOKIQ 2024/11/24-05:05:19.583314 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292934.sst): first pass time = 0.89's, 150.195'MB/sec 2024/11/24-05:05:20.908346 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292934.sst): old prealloc_size = 147639500, real_size = 32872848 2024/11/24-05:05:20.908886 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292934.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926212 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 116.91 avg-zval = 33.34 2024/11/24-05:05:20.909684 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292934.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:21.758532 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292934.sst): second pass time = 1.32's, 81.768'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.485'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3411.978'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 289.521'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32155.216'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.768'MB/sec zip pipeline throughput = 81.768'MB/sec entries = 926212 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.91 avg-zval = 33.34 usrkeys = 926212 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.91 avg-zval = 33.34 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926212 bytes = 3241760 } seq expand size = 6483484 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5129 value = 3.5067 dict = 1.00 all = 3.6306 } Zip/UnZip{ index = 0.0951 value = 0.2852 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 2252.702710 GB avg = 0.200 KB (by entry num) total key len = 307.194265 GB avg = 0.027 KB total ukey len = 237.577086 GB avg = 0.015 KB total ukey num = 15.749779101 Billion total entry num = 11.291093049 Billion write speed all = 35.255654264 MB/sec (with seq num) write speed all = 34.011620222 MB/sec (without seq num) 2024/11/24-05:05:21.805378 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292934: 926212 keys, 36969006 bytes, temperature: kWarm 2024/11/24-05:05:21.805448 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395921805413, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292934, "file_size": 36969006, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4451516301, "table_properties": {"data_size": 30879040, "index_size": 1762048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25933936, "raw_average_key_size": 28, "raw_value_size": 108283924, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926212, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395918, "oldest_key_time": 0, "file_creation_time": 1732395918, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292934, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:21.842520 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292935.sst) /dev/shm/ToplingTemp/Topling-276011-Sjri1Q 2024/11/24-05:05:22.772659 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292935.sst): first pass time = 0.93's, 144.305'MB/sec 2024/11/24-05:05:24.087996 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292935.sst): old prealloc_size = 147639500, real_size = 31210064 2024/11/24-05:05:24.088588 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292935.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920327 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.84 avg-zval = 31.73 2024/11/24-05:05:24.089342 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292935.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:24.899743 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292935.sst): second pass time = 1.31's, 82.502'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.332'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3113.761'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 276.609'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33034.279'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.502'MB/sec zip pipeline throughput = 82.502'MB/sec entries = 920327 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.84 avg-zval = 31.73 usrkeys = 920327 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.84 avg-zval = 31.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920327 bytes = 3106112 } seq expand size = 6442289 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.3764 value = 3.7132 dict = 1.00 all = 3.8158 } Zip/UnZip{ index = 0.0964 value = 0.2693 dict = 1.00 all = 0.2621 } ---------------------------- total value len = 2252.817601 GB avg = 0.200 KB (by entry num) total key len = 307.220034 GB avg = 0.027 KB total ukey len = 237.595493 GB avg = 0.015 KB total ukey num = 15.750699428 Billion total entry num = 11.292013376 Billion write speed all = 35.256066240 MB/sec (with seq num) write speed all = 34.011984618 MB/sec (without seq num) 2024/11/24-05:05:24.944678 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292935: 920327 keys, 35174216 bytes, temperature: kWarm 2024/11/24-05:05:24.944748 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395924944713, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292935, "file_size": 35174216, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4343951823, "largest_seqno": 4451516989, "table_properties": {"data_size": 29205888, "index_size": 1773888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25769156, "raw_average_key_size": 28, "raw_value_size": 108448614, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920327, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395921, "oldest_key_time": 0, "file_creation_time": 1732395921, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292935, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:24.987602 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292936.sst) /dev/shm/ToplingTemp/Topling-276011-gWmJdT 2024/11/24-05:05:26.017821 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292936.sst): first pass time = 1.03's, 130.286'MB/sec 2024/11/24-05:05:27.352084 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292936.sst): old prealloc_size = 147639500, real_size = 32887472 2024/11/24-05:05:27.352656 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292936.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930570 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.23 avg-zval = 33.19 2024/11/24-05:05:27.353435 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292936.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:28.210728 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292936.sst): second pass time = 1.33's, 81.117'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.087'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3229.587'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 292.551'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 24782.371'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.117'MB/sec zip pipeline throughput = 81.117'MB/sec entries = 930570 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.23 avg-zval = 33.19 usrkeys = 930570 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.23 avg-zval = 33.19 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930570 bytes = 3257008 } seq expand size = 6513990 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5135 value = 3.5022 dict = 1.00 all = 3.6279 } Zip/UnZip{ index = 0.0951 value = 0.2855 dict = 1.00 all = 0.2756 } ---------------------------- total value len = 2252.932277 GB avg = 0.199 KB (by entry num) total key len = 307.246090 GB avg = 0.027 KB total ukey len = 237.614104 GB avg = 0.015 KB total ukey num = 15.751629998 Billion total entry num = 11.292943946 Billion write speed all = 35.256396738 MB/sec (with seq num) write speed all = 34.012269321 MB/sec (without seq num) 2024/11/24-05:05:28.257315 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292936: 930570 keys, 36995582 bytes, temperature: kWarm 2024/11/24-05:05:28.257390 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395928257349, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292936, "file_size": 36995582, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4451515897, "table_properties": {"data_size": 30884384, "index_size": 1770240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26055960, "raw_average_key_size": 28, "raw_value_size": 108161808, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930570, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395924, "oldest_key_time": 0, "file_creation_time": 1732395924, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292936, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:28.308207 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292937.sst) /dev/shm/ToplingTemp/Topling-276011-qaUTWS 2024/11/24-05:05:29.129807 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292937.sst): first pass time = 0.82's, 163.368'MB/sec 2024/11/24-05:05:30.431506 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292937.sst): old prealloc_size = 147639500, real_size = 31479856 2024/11/24-05:05:30.432096 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292937.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.07 2024/11/24-05:05:30.432822 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292937.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:31.251043 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292937.sst): second pass time = 1.30's, 83.431'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.248'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3117.973'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 280.656'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30266.116'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.431'MB/sec zip pipeline throughput = 83.431'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 32.07 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 32.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.3779 value = 3.6799 dict = 1.00 all = 3.7870 } Zip/UnZip{ index = 0.0964 value = 0.2717 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 2253.047190 GB avg = 0.199 KB (by entry num) total key len = 307.271830 GB avg = 0.027 KB total ukey len = 237.632490 GB avg = 0.015 KB total ukey num = 15.752549298 Billion total entry num = 11.293863246 Billion write speed all = 35.256857539 MB/sec (with seq num) write speed all = 34.012680936 MB/sec (without seq num) 2024/11/24-05:05:31.295614 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292937: 919300 keys, 35441714 bytes, temperature: kWarm 2024/11/24-05:05:31.295686 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395931295650, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292937, "file_size": 35441714, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4325645046, "largest_seqno": 4436887098, "table_properties": {"data_size": 29478176, "index_size": 1771648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395928, "oldest_key_time": 0, "file_creation_time": 1732395928, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292937, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:31.335884 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292938.sst) /dev/shm/ToplingTemp/Topling-276011-FEvs4O 2024/11/24-05:05:32.358904 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292938.sst): first pass time = 1.02's, 131.203'MB/sec 2024/11/24-05:05:33.695903 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292938.sst): old prealloc_size = 147639500, real_size = 32609136 2024/11/24-05:05:33.696497 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292938.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931747 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 116.05 avg-zval = 32.85 2024/11/24-05:05:33.697288 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292938.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:34.552853 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292938.sst): second pass time = 1.34's, 80.919'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.852'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3100.982'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 288.223'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 29297.222'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.919'MB/sec zip pipeline throughput = 80.919'MB/sec entries = 931747 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.05 avg-zval = 32.85 usrkeys = 931747 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.05 avg-zval = 32.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931747 bytes = 3261136 } seq expand size = 6522229 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5139 value = 3.5332 dict = 1.00 all = 3.6551 } Zip/UnZip{ index = 0.0951 value = 0.2830 dict = 1.00 all = 0.2736 } ---------------------------- total value len = 2253.161841 GB avg = 0.199 KB (by entry num) total key len = 307.297919 GB avg = 0.027 KB total ukey len = 237.651125 GB avg = 0.015 KB total ukey num = 15.753481045 Billion total entry num = 11.294794993 Billion write speed all = 35.257192530 MB/sec (with seq num) write speed all = 34.012969854 MB/sec (without seq num) 2024/11/24-05:05:34.599453 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292938: 931747 keys, 36721178 bytes, temperature: kWarm 2024/11/24-05:05:34.599525 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395934599488, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292938, "file_size": 36721178, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4451517008, "table_properties": {"data_size": 30603616, "index_size": 1772416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26088916, "raw_average_key_size": 28, "raw_value_size": 108128842, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931747, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395931, "oldest_key_time": 0, "file_creation_time": 1732395931, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292938, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:34.637620 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292939.sst) /dev/shm/ToplingTemp/Topling-276011-WLTpRP 2024/11/24-05:05:35.629114 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292939.sst): first pass time = 0.99's, 135.374'MB/sec 2024/11/24-05:05:36.921747 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292939.sst): old prealloc_size = 147639500, real_size = 31468928 2024/11/24-05:05:36.922303 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292939.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.07 2024/11/24-05:05:36.923051 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292939.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:37.736395 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292939.sst): second pass time = 1.29's, 83.994'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.494'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3313.242'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 280.162'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32871.122'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.994'MB/sec zip pipeline throughput = 83.994'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.07 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4439 value = 3.6799 dict = 1.00 all = 3.7882 } Zip/UnZip{ index = 0.0957 value = 0.2717 dict = 1.00 all = 0.2640 } ---------------------------- total value len = 2253.276753 GB avg = 0.199 KB (by entry num) total key len = 307.323660 GB avg = 0.027 KB total ukey len = 237.669511 GB avg = 0.015 KB total ukey num = 15.754400345 Billion total entry num = 11.295714293 Billion write speed all = 35.257583719 MB/sec (with seq num) write speed all = 34.013314319 MB/sec (without seq num) 2024/11/24-05:05:37.779215 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292939: 919300 keys, 35430770 bytes, temperature: kWarm 2024/11/24-05:05:37.779293 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395937779257, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292939, "file_size": 35430770, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4306732653, "largest_seqno": 4417365791, "table_properties": {"data_size": 29478448, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395934, "oldest_key_time": 0, "file_creation_time": 1732395934, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292939, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:37.817132 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292940.sst) /dev/shm/ToplingTemp/Topling-276011-zjqiCS 2024/11/24-05:05:38.825480 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292940.sst): first pass time = 1.01's, 133.111'MB/sec 2024/11/24-05:05:40.175522 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292940.sst): old prealloc_size = 147639500, real_size = 31903440 2024/11/24-05:05:40.176103 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292940.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931764 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0299 GB avg-val = 116.05 avg-zval = 32.09 2024/11/24-05:05:40.176920 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292940.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:41.021108 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292940.sst): second pass time = 1.35's, 80.159'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.594'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3235.389'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 279.543'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34006.877'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.159'MB/sec zip pipeline throughput = 80.159'MB/sec entries = 931764 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.05 avg-zval = 32.09 usrkeys = 931764 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.05 avg-zval = 32.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931764 bytes = 3261184 } seq expand size = 6522348 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0360 GB } UnZip/Zip{ index = 10.5129 value = 3.6166 dict = 1.00 all = 3.7267 } Zip/UnZip{ index = 0.0951 value = 0.2765 dict = 1.00 all = 0.2683 } ---------------------------- total value len = 2253.391404 GB avg = 0.199 KB (by entry num) total key len = 307.349749 GB avg = 0.027 KB total ukey len = 237.688146 GB avg = 0.015 KB total ukey num = 15.755332109 Billion total entry num = 11.296646057 Billion write speed all = 35.257926959 MB/sec (with seq num) write speed all = 34.013611200 MB/sec (without seq num) 2024/11/24-05:05:41.065037 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292940: 931764 keys, 36015638 bytes, temperature: kWarm 2024/11/24-05:05:41.065107 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395941065071, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292940, "file_size": 36015638, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679976, "largest_seqno": 4451517019, "table_properties": {"data_size": 29897728, "index_size": 1772608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26089392, "raw_average_key_size": 28, "raw_value_size": 108128420, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931764, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395937, "oldest_key_time": 0, "file_creation_time": 1732395937, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292940, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:41.108779 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292941.sst) /dev/shm/ToplingTemp/Topling-276011-13ySeS 2024/11/24-05:05:42.098235 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292941.sst): first pass time = 0.99's, 135.653'MB/sec 2024/11/24-05:05:43.423776 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292941.sst): old prealloc_size = 147639500, real_size = 31327552 2024/11/24-05:05:43.424334 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292941.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.91 2024/11/24-05:05:43.425109 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292941.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-05:05:44.234371 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292941.sst): second pass time = 1.32's, 81.884'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.537'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3294.612'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 278.914'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33817.791'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.884'MB/sec zip pipeline throughput = 81.884'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.91 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4462 value = 3.6976 dict = 1.00 all = 3.8033 } Zip/UnZip{ index = 0.0957 value = 0.2704 dict = 1.00 all = 0.2629 } ---------------------------- total value len = 2253.506316 GB avg = 0.199 KB (by entry num) total key len = 307.375489 GB avg = 0.027 KB total ukey len = 237.706532 GB avg = 0.015 KB total ukey num = 15.756251409 Billion total entry num = 11.297565357 Billion write speed all = 35.258303650 MB/sec (with seq num) write speed all = 34.013941684 MB/sec (without seq num) 2024/11/24-05:05:44.277698 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292941: 919300 keys, 35289394 bytes, temperature: kWarm 2024/11/24-05:05:44.277764 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395944277730, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292941, "file_size": 35289394, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4287953642, "largest_seqno": 4397436900, "table_properties": {"data_size": 29337456, "index_size": 1760064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395941, "oldest_key_time": 0, "file_creation_time": 1732395941, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292941, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:44.316522 139957189985856 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292942.sst) /dev/shm/ToplingTemp/Topling-276011-iBEi9P 2024/11/24-05:05:45.085593 139957189985856 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292942.sst): first pass time = 0.77's, 164.944'MB/sec 2024/11/24-05:05:46.308572 139957189985856 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292942.sst): old prealloc_size = 147639500, real_size = 28626144 2024/11/24-05:05:46.309102 139957189985856 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292942.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 880886 prefix = 4 raw-key = 0.0141 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1022 GB zip-val = 0.0267 GB avg-val = 116.00 avg-zval = 30.33 2024/11/24-05:05:46.309814 139957189985856 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292942.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0022 GB, waited 0.000 sec, Key+Value = 0.127 GB 2024/11/24-05:05:47.091374 139957189985856 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00/292942.sst): second pass time = 1.22's, 83.601'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 41.258'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3326.488'MB/sec (index lex order gen) rebuild zvType time = 0.66's, 0.333'MB/sec write SST data time = 0.12's, 269.075'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.023'MB, throughput = 30347.715'MB/sec zip my value time = 1.22's, unzip length = 0.102'GB zip my value throughput = 83.601'MB/sec zip pipeline throughput = 83.601'MB/sec entries = 880886 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.00 avg-zval = 30.33 usrkeys = 880886 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.00 avg-zval = 30.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 880886 bytes = 2862896 } seq expand size = 6166202 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0176 GB value = 0.1022 GB dict = 1.02 MB all = 0.1268 GB } __ZipSize{ index = 0.0017 GB value = 0.0267 GB dict = 1.02 MB all = 0.0323 GB } UnZip/Zip{ index = 10.4438 value = 3.8244 dict = 1.00 all = 3.9277 } Zip/UnZip{ index = 0.0958 value = 0.2615 dict = 1.00 all = 0.2546 } ---------------------------- total value len = 2253.614665 GB avg = 0.199 KB (by entry num) total key len = 307.400154 GB avg = 0.027 KB total ukey len = 237.724150 GB avg = 0.015 KB total ukey num = 15.757132295 Billion total entry num = 11.298446243 Billion write speed all = 35.258748072 MB/sec (with seq num) write speed all = 34.014338030 MB/sec (without seq num) 2024/11/24-05:05:47.132319 139957189985856 [compaction_job.cc:1948] [default] [JOB 41943] Generated table #292942: 880886 keys, 32295310 bytes, temperature: kWarm 2024/11/24-05:05:47.132391 139957189985856 EVENT_LOG_v1 {"time_micros": 1732395947132355, "cf_name": "default", "job": 41943, "event": "table_file_creation", "file_number": 292942, "file_size": 32295310, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4397436901, "largest_seqno": 4451517031, "table_properties": {"data_size": 26718800, "index_size": 1686912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 24664808, "raw_average_key_size": 28, "raw_value_size": 102182524, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 880886, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732395944, "oldest_key_time": 0, "file_creation_time": 1732395944, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 292942, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-05:05:47.157155 139957189985856 [compaction_job.cc:703] job-41943: subcompact[0], size: 0.709451 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-41943/att-00 [292923,292924,292925,292926,292927,292928,292929,292930,292931,292932,292933,292934,292935,292936,292937,292938,292939,292940,292941,292942] 2024/11/24-05:05:47.281185 139957189985856 (Original Log Time 2024/11/24-05:04:41.225161) EVENT_LOG_v1 {"time_micros": 1732395881225135, "job": 41943, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [292870], "files_L3": [292920, 292836, 292837, 292838, 292839, 292840, 292841, 292842, 292791, 292792, 292793, 292794, 292795, 292796, 292797, 292798, 292799, 292800, 292801, 292802, 292803], "score": 0.975581, "input_data_size": 732269438, "oldest_snapshot_seqno": 4451946998} 2024/11/24-05:05:47.281189 139957189985856 (Original Log Time 2024/11/24-05:05:47.279579) [compaction_job.cc:2022] [default] [JOB 41943] Compacted 1@2 + 21@3 files to L3 => 709451102 bytes 2024/11/24-05:05:47.281191 139957189985856 (Original Log Time 2024/11/24-05:05:47.281112) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 11.1 rd, 10.8 wr, level 3, files in(1, 21) out(20 +0 blob) MB in(27.1, 671.3 +0.0 blob) out(676.6 +0.0 blob), read-write-amplify(50.8) write-amplify(25.0) OK, records in: 18576730, records dropped: 105598 output_compression: Snappy 2024/11/24-05:05:47.281194 139957189985856 (Original Log Time 2024/11/24-05:05:47.281159) EVENT_LOG_v1 {"time_micros": 1732395947281137, "cf": "default", "job": 41943, "event": "compaction_finished", "compaction_time_micros": 65932002, "compaction_time_cpu_micros": 28135712, "output_level": 3, "num_output_files": 20, "total_output_size": 709451102, "num_input_records": 18576730, "num_output_records": 18471132, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 497200154, "file_range_sync_nanos": 0, "file_fsync_nanos": 14633, "file_prepare_write_nanos": 47215, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/24-05:05:47.422765 139957189985856 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732395881,"executesMs":66191,"compactionJobId":41943,"attempt":0,"compactionInputRawBytes":2691980668,"compactionInputZipBytes":732269438,"compactionOutputRawBytes":0,"compactionOutputZipBytes":709451102,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-eca919a6953d0b2f708db024b7133918-8620eafe6e27349f-00"}}, response = 2024/11/24-05:05:47.422869 139957189985856 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-41943/att-00: olev 3, work 66.192 s, result 7.740 ms, install 1.630 ms, input{raw 2.507 GiB zip 698.347 MiB}