2024/11/24-06:18:14.726601 139957659977280 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00: fork to child time = 0.007613 sec 2024/11/24-06:18:14.727432 139957659977280 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00 2024/11/24-06:18:14.734583 139957659977280 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00 2024/11/24-06:18:14.735482 139957659977280 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-06:18:14.735717 139957659977280 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-06:18:14.790909 139957659977280 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42731/att-00/MANIFEST-206347 2024/11/24-06:18:14.819870 139957659977280 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42731/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 294735, last_sequence is 4475862137, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 294542 2024/11/24-06:18:14.819880 139957659977280 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-06:18:14.820029 139957659977280 [version_set.cc:5721] Creating manifest 294735 2024/11/24-06:18:14.855372 139957659977280 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-06:18:14.876448 139957659977280 [compaction_job.cc:2437] [default] [JOB 42731] Compacting 1@2 + 24@3 files to L3, score 0.98, subcompactions 1 : 1 2024/11/24-06:18:14.876463 139957659977280 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 32032 Base level 2, inputs: [294689(27MB)], [294709(35MB) 294710(6856KB) 294610(33MB) 294611(34MB) 294612(35MB) 294613(33MB) 294614(35MB) 294615(33MB) 294616(35MB) 294617(33MB) 294618(33MB) 294619(35MB) 294620(33MB) 294623(35MB) 294624(33MB) 294625(34MB) 294626(34MB) 294627(33MB) 294628(35MB) 294629(33MB) 294630(35MB) 294631(30MB) 294632(30MB) 294711(34MB)] 2024/11/24-06:18:14.884858 139957659977280 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:18:14.884888 139957659977280 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:18:14.884907 139957659977280 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:18:14.884912 139957659977280 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-06:18:14.884923 139957659977280 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-06:18:15.567595 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294736.sst) /dev/shm/ToplingTemp/Topling-287636-OphH2Q 2024/11/24-06:18:16.105997 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294736.sst): first pass time = 0.54's, 249.309'MB/sec 2024/11/24-06:18:17.458289 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294736.sst): old prealloc_size = 147639500, real_size = 33424560 2024/11/24-06:18:17.458857 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294736.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927867 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.65 avg-zval = 33.87 2024/11/24-06:18:17.459660 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294736.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:18.323403 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294736.sst): second pass time = 1.35's, 80.093'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.378'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3228.023'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.314'MB/sec write SST data time = 0.13's, 299.636'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33865.429'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.093'MB/sec zip pipeline throughput = 80.093'MB/sec entries = 927867 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.65 avg-zval = 33.87 usrkeys = 927867 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.65 avg-zval = 33.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927867 bytes = 3247552 } seq expand size = 6495069 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5122 value = 3.4441 dict = 1.00 all = 3.5766 } Zip/UnZip{ index = 0.0951 value = 0.2904 dict = 1.00 all = 0.2796 } ---------------------------- total value len = 2251.440306 GB avg = 0.200 KB (by entry num) total key len = 306.908943 GB avg = 0.027 KB total ukey len = 237.373285 GB avg = 0.015 KB total ukey num = 15.739589030 Billion total entry num = 11.280902978 Billion write speed all = 33.231310041 MB/sec (with seq num) write speed all = 32.059056692 MB/sec (without seq num) 2024/11/24-06:18:18.374997 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294736: 927867 keys, 37526136 bytes, temperature: kWarm 2024/11/24-06:18:18.375098 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400298375036, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294736, "file_size": 37526136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4475432183, "table_properties": {"data_size": 31427104, "index_size": 1765312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25980276, "raw_average_key_size": 28, "raw_value_size": 108237518, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927867, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400295, "oldest_key_time": 0, "file_creation_time": 1732400295, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294736, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:18.412470 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294737.sst) /dev/shm/ToplingTemp/Topling-287636-G40J8U 2024/11/24-06:18:19.481772 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294737.sst): first pass time = 1.07's, 125.523'MB/sec 2024/11/24-06:18:20.775081 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294737.sst): old prealloc_size = 147639500, real_size = 31249968 2024/11/24-06:18:20.775659 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294737.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921842 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.60 avg-zval = 31.73 2024/11/24-06:18:20.776600 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294737.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:21.594301 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294737.sst): second pass time = 1.29's, 83.876'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.132'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3185.521'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.12's, 285.275'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 35625.185'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.876'MB/sec zip pipeline throughput = 83.876'MB/sec entries = 921842 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.60 avg-zval = 31.73 usrkeys = 921842 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.60 avg-zval = 31.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921842 bytes = 3226464 } seq expand size = 6452894 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.7057 dict = 1.00 all = 3.7986 } Zip/UnZip{ index = 0.0957 value = 0.2699 dict = 1.00 all = 0.2633 } ---------------------------- total value len = 2251.555165 GB avg = 0.200 KB (by entry num) total key len = 306.934754 GB avg = 0.027 KB total ukey len = 237.391722 GB avg = 0.015 KB total ukey num = 15.740510872 Billion total entry num = 11.281824820 Billion write speed all = 33.231725350 MB/sec (with seq num) write speed all = 32.059426015 MB/sec (without seq num) 2024/11/24-06:18:21.638133 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294737: 921842 keys, 35333476 bytes, temperature: kWarm 2024/11/24-06:18:21.638206 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400301638169, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294737, "file_size": 35333476, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4339338575, "largest_seqno": 4475430012, "table_properties": {"data_size": 29254240, "index_size": 1765056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25811576, "raw_average_key_size": 28, "raw_value_size": 108406212, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921842, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400298, "oldest_key_time": 0, "file_creation_time": 1732400298, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294737, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:21.673886 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294738.sst) /dev/shm/ToplingTemp/Topling-287636-tq9Z5S 2024/11/24-06:18:22.267914 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294738.sst): first pass time = 0.59's, 52.479'MB/sec 2024/11/24-06:18:22.567847 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294738.sst): old prealloc_size = 147639500, real_size = 8093168 2024/11/24-06:18:22.568036 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294738.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 219525 prefix = 4 raw-key = 0.0035 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0250 GB zip-val = 0.0076 GB avg-val = 114.00 avg-zval = 34.70 2024/11/24-06:18:22.568237 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294738.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.031 GB 2024/11/24-06:18:22.761591 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294738.sst): second pass time = 0.30's, 83.662'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 46.380'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2559.192'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.335'MB/sec write SST data time = 0.03's, 300.080'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.259'MB, throughput = 8660.932'MB/sec zip my value time = 0.30's, unzip length = 0.025'GB zip my value throughput = 83.662'MB/sec zip pipeline throughput = 83.662'MB/sec entries = 219525 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.00 avg-zval = 34.70 usrkeys = 219525 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.00 avg-zval = 34.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 219525 bytes = 686032 } seq expand size = 1536675 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0044 GB value = 0.0250 GB dict = 0.26 MB all = 0.0312 GB } __ZipSize{ index = 0.0004 GB value = 0.0076 GB dict = 0.26 MB all = 0.0090 GB } UnZip/Zip{ index = 10.4147 value = 3.2857 dict = 1.00 all = 3.4695 } Zip/UnZip{ index = 0.0960 value = 0.3043 dict = 1.00 all = 0.2882 } ---------------------------- total value len = 2251.581727 GB avg = 0.200 KB (by entry num) total key len = 306.940901 GB avg = 0.027 KB total ukey len = 237.396112 GB avg = 0.015 KB total ukey num = 15.740730397 Billion total entry num = 11.282044345 Billion write speed all = 33.231646354 MB/sec (with seq num) write speed all = 32.059341982 MB/sec (without seq num) 2024/11/24-06:18:22.778270 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294738: 219525 keys, 8984763 bytes, temperature: kWarm 2024/11/24-06:18:22.778327 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400302778297, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294738, "file_size": 8984763, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4443453294, "largest_seqno": 4475430627, "table_properties": {"data_size": 7616544, "index_size": 421568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6146700, "raw_average_key_size": 28, "raw_value_size": 25025850, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 219525, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400301, "oldest_key_time": 0, "file_creation_time": 1732400301, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294738, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:22.802884 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294739.sst) /dev/shm/ToplingTemp/Topling-287636-ufoMqS 2024/11/24-06:18:23.803720 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294739.sst): first pass time = 1.00's, 134.110'MB/sec 2024/11/24-06:18:25.114102 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294739.sst): old prealloc_size = 147639500, real_size = 31398672 2024/11/24-06:18:25.114650 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294739.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.99 2024/11/24-06:18:25.115427 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294739.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:25.934099 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294739.sst): second pass time = 1.31's, 82.831'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.264'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3368.389'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.12's, 286.679'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 28805.320'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.831'MB/sec zip pipeline throughput = 82.831'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.99 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6886 dict = 1.00 all = 3.7834 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2251.696640 GB avg = 0.200 KB (by entry num) total key len = 306.966641 GB avg = 0.027 KB total ukey len = 237.414498 GB avg = 0.015 KB total ukey num = 15.741649697 Billion total entry num = 11.282963645 Billion write speed all = 33.232103865 MB/sec (with seq num) write speed all = 32.059752278 MB/sec (without seq num) 2024/11/24-06:18:25.978462 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294739: 919300 keys, 35475378 bytes, temperature: kWarm 2024/11/24-06:18:25.978531 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400305978496, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294739, "file_size": 35475378, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4414357721, "table_properties": {"data_size": 29408512, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400302, "oldest_key_time": 0, "file_creation_time": 1732400302, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294739, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:26.019685 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294740.sst) /dev/shm/ToplingTemp/Topling-287636-ml3qDS 2024/11/24-06:18:27.019545 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294740.sst): first pass time = 1.00's, 134.242'MB/sec 2024/11/24-06:18:28.362963 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294740.sst): old prealloc_size = 147639500, real_size = 32744480 2024/11/24-06:18:28.363526 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294740.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927830 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.66 avg-zval = 33.14 2024/11/24-06:18:28.364297 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294740.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:29.208783 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294740.sst): second pass time = 1.34's, 80.635'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.564'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3388.029'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 292.417'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32550.589'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.635'MB/sec zip pipeline throughput = 80.635'MB/sec entries = 927830 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 33.14 usrkeys = 927830 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 33.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927830 bytes = 3247424 } seq expand size = 6494810 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5122 value = 3.5203 dict = 1.00 all = 3.6427 } Zip/UnZip{ index = 0.0951 value = 0.2841 dict = 1.00 all = 0.2745 } ---------------------------- total value len = 2251.811373 GB avg = 0.200 KB (by entry num) total key len = 306.992621 GB avg = 0.027 KB total ukey len = 237.433055 GB avg = 0.015 KB total ukey num = 15.742577527 Billion total entry num = 11.283891475 Billion write speed all = 33.232518009 MB/sec (with seq num) write speed all = 32.060119880 MB/sec (without seq num) 2024/11/24-06:18:29.255710 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294740: 927830 keys, 36845772 bytes, temperature: kWarm 2024/11/24-06:18:29.255781 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400309255745, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294740, "file_size": 36845772, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178655, "largest_seqno": 4475431553, "table_properties": {"data_size": 30747088, "index_size": 1765248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979240, "raw_average_key_size": 28, "raw_value_size": 108238504, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927830, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400306, "oldest_key_time": 0, "file_creation_time": 1732400306, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294740, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:29.292619 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294741.sst) /dev/shm/ToplingTemp/Topling-287636-HA4qeT 2024/11/24-06:18:30.234588 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294741.sst): first pass time = 0.94's, 142.492'MB/sec 2024/11/24-06:18:31.490912 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294741.sst): old prealloc_size = 147639500, real_size = 31440368 2024/11/24-06:18:31.491475 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294741.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.04 2024/11/24-06:18:31.492251 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294741.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:32.316836 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294741.sst): second pass time = 1.26's, 86.399'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.004'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3236.356'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 278.064'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 27932.499'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 86.399'MB/sec zip pipeline throughput = 86.399'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.04 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6834 dict = 1.00 all = 3.7790 } Zip/UnZip{ index = 0.0957 value = 0.2715 dict = 1.00 all = 0.2646 } ---------------------------- total value len = 2251.926286 GB avg = 0.200 KB (by entry num) total key len = 307.018361 GB avg = 0.027 KB total ukey len = 237.451441 GB avg = 0.015 KB total ukey num = 15.743496827 Billion total entry num = 11.284810775 Billion write speed all = 33.233003259 MB/sec (with seq num) write speed all = 32.060556942 MB/sec (without seq num) 2024/11/24-06:18:32.361041 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294741: 919300 keys, 35517106 bytes, temperature: kWarm 2024/11/24-06:18:32.361129 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400312361092, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294741, "file_size": 35517106, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4302106462, "largest_seqno": 4453598768, "table_properties": {"data_size": 29450208, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400309, "oldest_key_time": 0, "file_creation_time": 1732400309, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294741, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:32.398031 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294742.sst) /dev/shm/ToplingTemp/Topling-287636-YQ8BGS 2024/11/24-06:18:33.275708 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294742.sst): first pass time = 0.88's, 152.930'MB/sec 2024/11/24-06:18:34.632580 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294742.sst): old prealloc_size = 147639500, real_size = 33369680 2024/11/24-06:18:34.633124 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294742.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927508 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.71 avg-zval = 33.82 2024/11/24-06:18:34.633889 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294742.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:35.489730 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294742.sst): second pass time = 1.36's, 79.847'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.716'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3380.590'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 296.980'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34533.674'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.847'MB/sec zip pipeline throughput = 79.847'MB/sec entries = 927508 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.71 avg-zval = 33.82 usrkeys = 927508 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.71 avg-zval = 33.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927508 bytes = 3246288 } seq expand size = 6492556 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5108 value = 3.4504 dict = 1.00 all = 3.5820 } Zip/UnZip{ index = 0.0951 value = 0.2898 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 2252.041026 GB avg = 0.200 KB (by entry num) total key len = 307.044331 GB avg = 0.027 KB total ukey len = 237.469991 GB avg = 0.015 KB total ukey num = 15.744424335 Billion total entry num = 11.285738283 Billion write speed all = 33.233461236 MB/sec (with seq num) write speed all = 32.060966869 MB/sec (without seq num) 2024/11/24-06:18:35.535991 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294742: 927508 keys, 37470290 bytes, temperature: kWarm 2024/11/24-06:18:35.536061 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400315536025, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294742, "file_size": 37470290, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4475430432, "table_properties": {"data_size": 31372736, "index_size": 1764864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25970224, "raw_average_key_size": 28, "raw_value_size": 108247620, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927508, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400312, "oldest_key_time": 0, "file_creation_time": 1732400312, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294742, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:35.572486 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294743.sst) /dev/shm/ToplingTemp/Topling-287636-lpT6XS 2024/11/24-06:18:36.585354 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294743.sst): first pass time = 1.01's, 132.517'MB/sec 2024/11/24-06:18:37.902906 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294743.sst): old prealloc_size = 147639500, real_size = 31152192 2024/11/24-06:18:37.903465 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294743.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922678 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.47 avg-zval = 31.60 2024/11/24-06:18:37.904201 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294743.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:38.725186 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294743.sst): second pass time = 1.32's, 82.309'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.715'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3292.209'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 277.997'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32561.748'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.309'MB/sec zip pipeline throughput = 82.309'MB/sec entries = 922678 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.47 avg-zval = 31.60 usrkeys = 922678 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.47 avg-zval = 31.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922678 bytes = 3114048 } seq expand size = 6458746 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4440 value = 3.7175 dict = 1.00 all = 3.8214 } Zip/UnZip{ index = 0.0957 value = 0.2690 dict = 1.00 all = 0.2617 } ---------------------------- total value len = 2252.155867 GB avg = 0.200 KB (by entry num) total key len = 307.070166 GB avg = 0.027 KB total ukey len = 237.488444 GB avg = 0.015 KB total ukey num = 15.745347013 Billion total entry num = 11.286660961 Billion write speed all = 33.233891732 MB/sec (with seq num) write speed all = 32.061350773 MB/sec (without seq num) 2024/11/24-06:18:38.769333 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294743: 922678 keys, 35122702 bytes, temperature: kWarm 2024/11/24-06:18:38.769407 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400318769373, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294743, "file_size": 35122702, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4344428035, "largest_seqno": 4475429763, "table_properties": {"data_size": 29154416, "index_size": 1766912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25834984, "raw_average_key_size": 28, "raw_value_size": 108382748, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922678, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400315, "oldest_key_time": 0, "file_creation_time": 1732400315, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294743, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:38.809209 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294744.sst) /dev/shm/ToplingTemp/Topling-287636-Vv2CnU 2024/11/24-06:18:39.762655 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294744.sst): first pass time = 0.95's, 140.776'MB/sec 2024/11/24-06:18:41.089919 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294744.sst): old prealloc_size = 147639500, real_size = 32911696 2024/11/24-06:18:41.090446 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294744.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924315 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.21 avg-zval = 33.45 2024/11/24-06:18:41.091228 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294744.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:41.938466 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294744.sst): second pass time = 1.33's, 81.671'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.609'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3495.139'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 289.126'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32843.659'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.671'MB/sec zip pipeline throughput = 81.671'MB/sec entries = 924315 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.21 avg-zval = 33.45 usrkeys = 924315 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.21 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924315 bytes = 3235120 } seq expand size = 6470205 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5128 value = 3.5036 dict = 1.00 all = 3.6272 } Zip/UnZip{ index = 0.0951 value = 0.2854 dict = 1.00 all = 0.2757 } ---------------------------- total value len = 2252.270675 GB avg = 0.200 KB (by entry num) total key len = 307.096047 GB avg = 0.027 KB total ukey len = 237.506931 GB avg = 0.015 KB total ukey num = 15.746271328 Billion total entry num = 11.287585276 Billion write speed all = 33.234331913 MB/sec (with seq num) write speed all = 32.061743859 MB/sec (without seq num) 2024/11/24-06:18:41.983442 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294744: 924315 keys, 37003028 bytes, temperature: kWarm 2024/11/24-06:18:41.983521 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400321983487, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294744, "file_size": 37003028, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4475425803, "table_properties": {"data_size": 30921984, "index_size": 1758464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25880820, "raw_average_key_size": 28, "raw_value_size": 108336974, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924315, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400318, "oldest_key_time": 0, "file_creation_time": 1732400318, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294744, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:42.020765 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294745.sst) /dev/shm/ToplingTemp/Topling-287636-aMHpjT 2024/11/24-06:18:43.030374 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294745.sst): first pass time = 1.01's, 132.946'MB/sec 2024/11/24-06:18:44.399696 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294745.sst): old prealloc_size = 147639500, real_size = 31447168 2024/11/24-06:18:44.400284 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294745.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927758 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0294 GB avg-val = 116.67 avg-zval = 31.74 2024/11/24-06:18:44.401075 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294745.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:45.234726 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294745.sst): second pass time = 1.37's, 79.091'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 42.573'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3124.584'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.329'MB/sec write SST data time = 0.13's, 276.557'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34368.918'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.091'MB/sec zip pipeline throughput = 79.091'MB/sec entries = 927758 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.67 avg-zval = 31.74 usrkeys = 927758 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.67 avg-zval = 31.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927758 bytes = 3247168 } seq expand size = 6494306 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5117 value = 3.6754 dict = 1.00 all = 3.7756 } Zip/UnZip{ index = 0.0951 value = 0.2721 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2252.385410 GB avg = 0.200 KB (by entry num) total key len = 307.122024 GB avg = 0.027 KB total ukey len = 237.525486 GB avg = 0.015 KB total ukey num = 15.747199086 Billion total entry num = 11.288513034 Billion write speed all = 33.234736562 MB/sec (with seq num) write speed all = 32.062102322 MB/sec (without seq num) 2024/11/24-06:18:45.279662 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294745: 927758 keys, 35548352 bytes, temperature: kWarm 2024/11/24-06:18:45.279732 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400325279696, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294745, "file_size": 35548352, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4475431219, "table_properties": {"data_size": 29449840, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25977224, "raw_average_key_size": 28, "raw_value_size": 108240636, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927758, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400322, "oldest_key_time": 0, "file_creation_time": 1732400322, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294745, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:45.324797 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294746.sst) /dev/shm/ToplingTemp/Topling-287636-TAJ02T 2024/11/24-06:18:46.166224 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294746.sst): first pass time = 0.84's, 159.520'MB/sec 2024/11/24-06:18:47.479045 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294746.sst): old prealloc_size = 147639500, real_size = 31416240 2024/11/24-06:18:47.479590 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294746.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.00 2024/11/24-06:18:47.480343 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294746.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:48.294949 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294746.sst): second pass time = 1.31's, 82.675'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.503'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3388.970'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 276.216'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33494.166'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.675'MB/sec zip pipeline throughput = 82.675'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 32.00 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 32.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3783 value = 3.6879 dict = 1.00 all = 3.7815 } Zip/UnZip{ index = 0.0964 value = 0.2712 dict = 1.00 all = 0.2644 } ---------------------------- total value len = 2252.500322 GB avg = 0.200 KB (by entry num) total key len = 307.147765 GB avg = 0.027 KB total ukey len = 237.543872 GB avg = 0.015 KB total ukey num = 15.748118386 Billion total entry num = 11.289432334 Billion write speed all = 33.235242264 MB/sec (with seq num) write speed all = 32.062559127 MB/sec (without seq num) 2024/11/24-06:18:48.339654 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294746: 919300 keys, 35492978 bytes, temperature: kWarm 2024/11/24-06:18:48.339725 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400328339689, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294746, "file_size": 35492978, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4272717947, "largest_seqno": 4425119733, "table_properties": {"data_size": 29414624, "index_size": 1771584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400325, "oldest_key_time": 0, "file_creation_time": 1732400325, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294746, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:48.377147 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294747.sst) /dev/shm/ToplingTemp/Topling-287636-M38fkR 2024/11/24-06:18:48.918465 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294747.sst): first pass time = 0.54's, 247.966'MB/sec 2024/11/24-06:18:50.274494 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294747.sst): old prealloc_size = 147639500, real_size = 33171328 2024/11/24-06:18:50.275114 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294747.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927798 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.66 avg-zval = 33.59 2024/11/24-06:18:50.275870 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294747.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:51.126409 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294747.sst): second pass time = 1.35's, 79.893'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.754'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2980.007'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 293.054'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32136.019'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.893'MB/sec zip pipeline throughput = 79.893'MB/sec entries = 927798 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.66 avg-zval = 33.59 usrkeys = 927798 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.66 avg-zval = 33.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927798 bytes = 3247312 } seq expand size = 6494586 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.4440 value = 3.4734 dict = 1.00 all = 3.6010 } Zip/UnZip{ index = 0.0957 value = 0.2879 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 2252.615056 GB avg = 0.200 KB (by entry num) total key len = 307.173743 GB avg = 0.027 KB total ukey len = 237.562428 GB avg = 0.015 KB total ukey num = 15.749046184 Billion total entry num = 11.290360132 Billion write speed all = 33.235847419 MB/sec (with seq num) write speed all = 32.063111022 MB/sec (without seq num) 2024/11/24-06:18:51.172324 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294747: 927798 keys, 37272716 bytes, temperature: kWarm 2024/11/24-06:18:51.172408 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400331172359, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294747, "file_size": 37272716, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4475430672, "table_properties": {"data_size": 31162480, "index_size": 1776704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25978344, "raw_average_key_size": 28, "raw_value_size": 108239496, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927798, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400328, "oldest_key_time": 0, "file_creation_time": 1732400328, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294747, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:51.208967 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294748.sst) /dev/shm/ToplingTemp/Topling-287636-D6jJdU 2024/11/24-06:18:52.145947 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294748.sst): first pass time = 0.94's, 143.254'MB/sec 2024/11/24-06:18:53.458844 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294748.sst): old prealloc_size = 147639500, real_size = 31479200 2024/11/24-06:18:53.459454 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294748.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.08 2024/11/24-06:18:53.460224 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294748.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:54.287472 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294748.sst): second pass time = 1.31's, 82.674'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.911'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2992.763'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.329'MB/sec write SST data time = 0.13's, 273.303'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34904.616'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.674'MB/sec zip pipeline throughput = 82.674'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.08 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.08 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4439 value = 3.6786 dict = 1.00 all = 3.7748 } Zip/UnZip{ index = 0.0957 value = 0.2718 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2252.729969 GB avg = 0.200 KB (by entry num) total key len = 307.199483 GB avg = 0.027 KB total ukey len = 237.580814 GB avg = 0.015 KB total ukey num = 15.749965484 Billion total entry num = 11.291279432 Billion write speed all = 33.236309520 MB/sec (with seq num) write speed all = 32.063525770 MB/sec (without seq num) 2024/11/24-06:18:54.331434 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294748: 919300 keys, 35555890 bytes, temperature: kWarm 2024/11/24-06:18:54.331506 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400334331469, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294748, "file_size": 35555890, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4313538803, "largest_seqno": 4464765206, "table_properties": {"data_size": 29488720, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400331, "oldest_key_time": 0, "file_creation_time": 1732400331, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294748, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:54.374042 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294749.sst) /dev/shm/ToplingTemp/Topling-287636-4fqWPS 2024/11/24-06:18:55.250871 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294749.sst): first pass time = 0.88's, 153.079'MB/sec 2024/11/24-06:18:56.575107 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294749.sst): old prealloc_size = 147639500, real_size = 33327456 2024/11/24-06:18:56.575695 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294749.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928096 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.62 avg-zval = 33.76 2024/11/24-06:18:56.576463 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294749.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:18:57.434067 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294749.sst): second pass time = 1.32's, 81.781'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.576'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3139.887'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 292.969'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 28559.862'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.781'MB/sec zip pipeline throughput = 81.781'MB/sec entries = 928096 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.62 avg-zval = 33.76 usrkeys = 928096 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.62 avg-zval = 33.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928096 bytes = 3248352 } seq expand size = 6496672 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5133 value = 3.4546 dict = 1.00 all = 3.5859 } Zip/UnZip{ index = 0.0951 value = 0.2895 dict = 1.00 all = 0.2789 } ---------------------------- total value len = 2252.844696 GB avg = 0.200 KB (by entry num) total key len = 307.225470 GB avg = 0.027 KB total ukey len = 237.599376 GB avg = 0.015 KB total ukey num = 15.750893580 Billion total entry num = 11.292207528 Billion write speed all = 33.236778628 MB/sec (with seq num) write speed all = 32.063946393 MB/sec (without seq num) 2024/11/24-06:18:57.480383 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294749: 928096 keys, 37429004 bytes, temperature: kWarm 2024/11/24-06:18:57.480454 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400337480416, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294749, "file_size": 37429004, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4475431753, "table_properties": {"data_size": 31329680, "index_size": 1765568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25986688, "raw_average_key_size": 28, "raw_value_size": 108231136, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928096, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400334, "oldest_key_time": 0, "file_creation_time": 1732400334, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294749, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:18:57.519296 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294750.sst) /dev/shm/ToplingTemp/Topling-287636-oSGQXS 2024/11/24-06:18:58.509082 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294750.sst): first pass time = 0.99's, 135.609'MB/sec 2024/11/24-06:18:59.822449 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294750.sst): old prealloc_size = 147639500, real_size = 30917008 2024/11/24-06:18:59.823038 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294750.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923962 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 117.26 avg-zval = 31.30 2024/11/24-06:18:59.823794 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294750.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:19:00.638011 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294750.sst): second pass time = 1.31's, 82.575'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.783'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3128.236'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 274.156'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 35316.495'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.575'MB/sec zip pipeline throughput = 82.575'MB/sec entries = 923962 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.26 avg-zval = 31.30 usrkeys = 923962 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.26 avg-zval = 31.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923962 bytes = 3118384 } seq expand size = 6467734 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0349 GB } UnZip/Zip{ index = 10.4453 value = 3.7469 dict = 1.00 all = 3.8468 } Zip/UnZip{ index = 0.0957 value = 0.2669 dict = 1.00 all = 0.2600 } ---------------------------- total value len = 2252.959511 GB avg = 0.199 KB (by entry num) total key len = 307.251341 GB avg = 0.027 KB total ukey len = 237.617855 GB avg = 0.015 KB total ukey num = 15.751817542 Billion total entry num = 11.293131490 Billion write speed all = 33.237222586 MB/sec (with seq num) write speed all = 32.064343173 MB/sec (without seq num) 2024/11/24-06:19:00.681262 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294750: 923962 keys, 34890610 bytes, temperature: kWarm 2024/11/24-06:19:00.681331 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400340681296, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294750, "file_size": 34890610, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4352131795, "largest_seqno": 4475429829, "table_properties": {"data_size": 28916672, "index_size": 1769152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25870936, "raw_average_key_size": 28, "raw_value_size": 108346840, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923962, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400337, "oldest_key_time": 0, "file_creation_time": 1732400337, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294750, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:19:00.719557 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294751.sst) /dev/shm/ToplingTemp/Topling-287636-V2UXgV 2024/11/24-06:19:01.738995 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294751.sst): first pass time = 1.02's, 131.664'MB/sec 2024/11/24-06:19:03.092940 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294751.sst): old prealloc_size = 147639500, real_size = 32652304 2024/11/24-06:19:03.093516 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294751.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923011 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0307 GB avg-val = 117.41 avg-zval = 33.22 2024/11/24-06:19:03.094922 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294751.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:19:03.923573 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294751.sst): second pass time = 1.35's, 80.092'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.234'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3294.126'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.328'MB/sec write SST data time = 0.13's, 287.966'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 31297.930'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.092'MB/sec zip pipeline throughput = 80.092'MB/sec entries = 923011 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.41 avg-zval = 33.22 usrkeys = 923011 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.41 avg-zval = 33.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923011 bytes = 3230560 } seq expand size = 6461077 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5132 value = 3.5341 dict = 1.00 all = 3.6531 } Zip/UnZip{ index = 0.0951 value = 0.2830 dict = 1.00 all = 0.2737 } ---------------------------- total value len = 2253.074346 GB avg = 0.199 KB (by entry num) total key len = 307.277185 GB avg = 0.027 KB total ukey len = 237.636315 GB avg = 0.015 KB total ukey num = 15.752740553 Billion total entry num = 11.294054501 Billion write speed all = 33.237631191 MB/sec (with seq num) write speed all = 32.064705946 MB/sec (without seq num) 2024/11/24-06:19:03.967994 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294751: 923011 keys, 36740594 bytes, temperature: kWarm 2024/11/24-06:19:03.968062 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400343968025, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294751, "file_size": 36740594, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4475431619, "table_properties": {"data_size": 30665472, "index_size": 1755904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25844308, "raw_average_key_size": 28, "raw_value_size": 108373538, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923011, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400340, "oldest_key_time": 0, "file_creation_time": 1732400340, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294751, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:19:04.003416 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294752.sst) /dev/shm/ToplingTemp/Topling-287636-HJjKNQ 2024/11/24-06:19:04.968633 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294752.sst): first pass time = 0.97's, 139.060'MB/sec 2024/11/24-06:19:06.312277 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294752.sst): old prealloc_size = 147639500, real_size = 32233280 2024/11/24-06:19:06.312886 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294752.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927822 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0302 GB avg-val = 116.66 avg-zval = 32.58 2024/11/24-06:19:06.314324 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294752.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:19:07.160077 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294752.sst): second pass time = 1.34's, 80.602'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.859'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3030.461'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 281.515'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31688.173'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.602'MB/sec zip pipeline throughput = 80.602'MB/sec entries = 927822 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.66 avg-zval = 32.58 usrkeys = 927822 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.66 avg-zval = 32.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927822 bytes = 3247392 } seq expand size = 6494754 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.4466 value = 3.5811 dict = 1.00 all = 3.6939 } Zip/UnZip{ index = 0.0957 value = 0.2792 dict = 1.00 all = 0.2707 } ---------------------------- total value len = 2253.189079 GB avg = 0.199 KB (by entry num) total key len = 307.303164 GB avg = 0.027 KB total ukey len = 237.654872 GB avg = 0.015 KB total ukey num = 15.753668375 Billion total entry num = 11.294982323 Billion write speed all = 33.238061369 MB/sec (with seq num) write speed all = 32.065089050 MB/sec (without seq num) 2024/11/24-06:19:07.204156 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294752: 927822 keys, 36334572 bytes, temperature: kWarm 2024/11/24-06:19:07.204225 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400347204191, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294752, "file_size": 36334572, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4475431824, "table_properties": {"data_size": 30224816, "index_size": 1776320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979016, "raw_average_key_size": 28, "raw_value_size": 108238736, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927822, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400343, "oldest_key_time": 0, "file_creation_time": 1732400343, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294752, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:19:07.244999 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294753.sst) /dev/shm/ToplingTemp/Topling-287636-gfQ4pR 2024/11/24-06:19:08.249236 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294753.sst): first pass time = 1.00's, 133.657'MB/sec 2024/11/24-06:19:09.579393 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294753.sst): old prealloc_size = 147639500, real_size = 31316992 2024/11/24-06:19:09.579961 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294753.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.90 2024/11/24-06:19:09.581380 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294753.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:19:10.388884 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294753.sst): second pass time = 1.33's, 81.598'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.725'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3261.346'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 278.872'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32981.032'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.598'MB/sec zip pipeline throughput = 81.598'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.90 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4451 value = 3.6989 dict = 1.00 all = 3.7921 } Zip/UnZip{ index = 0.0957 value = 0.2703 dict = 1.00 all = 0.2637 } ---------------------------- total value len = 2253.303992 GB avg = 0.199 KB (by entry num) total key len = 307.328905 GB avg = 0.027 KB total ukey len = 237.673258 GB avg = 0.015 KB total ukey num = 15.754587675 Billion total entry num = 11.295901623 Billion write speed all = 33.238494059 MB/sec (with seq num) write speed all = 32.065475437 MB/sec (without seq num) 2024/11/24-06:19:10.432843 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294753: 919300 keys, 35393714 bytes, temperature: kWarm 2024/11/24-06:19:10.432913 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400350432878, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294753, "file_size": 35393714, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4288849106, "largest_seqno": 4440777199, "table_properties": {"data_size": 29326704, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400347, "oldest_key_time": 0, "file_creation_time": 1732400347, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294753, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:19:10.471259 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294754.sst) /dev/shm/ToplingTemp/Topling-287636-w8CjER 2024/11/24-06:19:11.443227 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294754.sst): first pass time = 0.97's, 138.094'MB/sec 2024/11/24-06:19:12.802697 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294754.sst): old prealloc_size = 147639500, real_size = 33381552 2024/11/24-06:19:12.803244 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294754.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927526 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.71 avg-zval = 33.83 2024/11/24-06:19:12.804045 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294754.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:19:13.668311 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294754.sst): second pass time = 1.36's, 79.667'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.303'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3392.737'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 292.289'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28067.896'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.667'MB/sec zip pipeline throughput = 79.667'MB/sec entries = 927526 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.71 avg-zval = 33.83 usrkeys = 927526 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.71 avg-zval = 33.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927526 bytes = 3246352 } seq expand size = 6492682 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4474 value = 3.4502 dict = 1.00 all = 3.5808 } Zip/UnZip{ index = 0.0957 value = 0.2898 dict = 1.00 all = 0.2793 } ---------------------------- total value len = 2253.418731 GB avg = 0.199 KB (by entry num) total key len = 307.354876 GB avg = 0.027 KB total ukey len = 237.691808 GB avg = 0.015 KB total ukey num = 15.755515201 Billion total entry num = 11.296829149 Billion write speed all = 33.238905621 MB/sec (with seq num) write speed all = 32.065840617 MB/sec (without seq num) 2024/11/24-06:19:13.714097 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294754: 927526 keys, 37482486 bytes, temperature: kWarm 2024/11/24-06:19:13.714164 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400353714129, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294754, "file_size": 37482486, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4475429856, "table_properties": {"data_size": 31373856, "index_size": 1775616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25970728, "raw_average_key_size": 28, "raw_value_size": 108247136, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927526, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400350, "oldest_key_time": 0, "file_creation_time": 1732400350, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294754, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:19:13.751113 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294755.sst) /dev/shm/ToplingTemp/Topling-287636-VPy79U 2024/11/24-06:19:14.706625 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294755.sst): first pass time = 0.96's, 140.473'MB/sec 2024/11/24-06:19:16.018175 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294755.sst): old prealloc_size = 147639500, real_size = 31284880 2024/11/24-06:19:16.018766 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294755.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921125 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0293 GB avg-val = 117.71 avg-zval = 31.80 2024/11/24-06:19:16.020760 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294755.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:19:16.833205 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294755.sst): second pass time = 1.31's, 82.722'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.394'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3116.012'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 277.587'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33891.603'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.722'MB/sec zip pipeline throughput = 82.722'MB/sec entries = 921125 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.71 avg-zval = 31.80 usrkeys = 921125 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.71 avg-zval = 31.80 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921125 bytes = 3223952 } seq expand size = 6447875 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4438 value = 3.7018 dict = 1.00 all = 3.7951 } Zip/UnZip{ index = 0.0958 value = 0.2701 dict = 1.00 all = 0.2635 } ---------------------------- total value len = 2253.533606 GB avg = 0.199 KB (by entry num) total key len = 307.380667 GB avg = 0.027 KB total ukey len = 237.710231 GB avg = 0.015 KB total ukey num = 15.756436326 Billion total entry num = 11.297750274 Billion write speed all = 33.239365979 MB/sec (with seq num) write speed all = 32.066253517 MB/sec (without seq num) 2024/11/24-06:19:16.876258 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294755: 921125 keys, 35366196 bytes, temperature: kWarm 2024/11/24-06:19:16.876329 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400356876293, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294755, "file_size": 35366196, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4335225060, "largest_seqno": 4475429876, "table_properties": {"data_size": 29290432, "index_size": 1763968, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25791500, "raw_average_key_size": 28, "raw_value_size": 108426318, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921125, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400353, "oldest_key_time": 0, "file_creation_time": 1732400353, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294755, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:19:16.918955 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294756.sst) /dev/shm/ToplingTemp/Topling-287636-G7K7UQ 2024/11/24-06:19:17.941720 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294756.sst): first pass time = 1.02's, 131.240'MB/sec 2024/11/24-06:19:19.284873 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294756.sst): old prealloc_size = 147639500, real_size = 33312144 2024/11/24-06:19:19.285483 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294756.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926078 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0313 GB avg-val = 116.93 avg-zval = 33.81 2024/11/24-06:19:19.287126 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294756.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:19:20.139616 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294756.sst): second pass time = 1.34's, 80.669'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.771'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3027.098'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 294.192'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33691.679'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.669'MB/sec zip pipeline throughput = 80.669'MB/sec entries = 926078 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.93 avg-zval = 33.81 usrkeys = 926078 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.93 avg-zval = 33.81 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926078 bytes = 3241280 } seq expand size = 6482546 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4461 value = 3.4589 dict = 1.00 all = 3.5876 } Zip/UnZip{ index = 0.0957 value = 0.2891 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 2253.648376 GB avg = 0.199 KB (by entry num) total key len = 307.406597 GB avg = 0.027 KB total ukey len = 237.728752 GB avg = 0.015 KB total ukey num = 15.757362404 Billion total entry num = 11.298676352 Billion write speed all = 33.239765697 MB/sec (with seq num) write speed all = 32.066607421 MB/sec (without seq num) 2024/11/24-06:19:20.184644 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294756: 926078 keys, 37411762 bytes, temperature: kWarm 2024/11/24-06:19:20.184715 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400360184680, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294756, "file_size": 37411762, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4475430029, "table_properties": {"data_size": 31307392, "index_size": 1773056, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25930184, "raw_average_key_size": 28, "raw_value_size": 108287684, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926078, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400356, "oldest_key_time": 0, "file_creation_time": 1732400356, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294756, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:19:20.221832 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294757.sst) /dev/shm/ToplingTemp/Topling-287636-M4vL7T 2024/11/24-06:19:21.173884 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294757.sst): first pass time = 0.95's, 140.983'MB/sec 2024/11/24-06:19:22.444007 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294757.sst): old prealloc_size = 147639500, real_size = 28206624 2024/11/24-06:19:22.444600 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294757.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 942759 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0262 GB avg-val = 114.37 avg-zval = 27.75 2024/11/24-06:19:22.446641 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294757.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:19:23.278214 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294757.sst): second pass time = 1.27's, 84.948'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 38.780'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3160.245'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 251.416'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.075'MB, throughput = 32730.223'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.948'MB/sec zip pipeline throughput = 84.948'MB/sec entries = 942759 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.37 avg-zval = 27.75 usrkeys = 942759 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.37 avg-zval = 27.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 942759 bytes = 3299664 } seq expand size = 6599313 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0262 GB dict = 1.08 MB all = 0.0323 GB } UnZip/Zip{ index = 10.4443 value = 4.1207 dict = 1.00 all = 4.1489 } Zip/UnZip{ index = 0.0957 value = 0.2427 dict = 1.00 all = 0.2410 } ---------------------------- total value len = 2253.762796 GB avg = 0.199 KB (by entry num) total key len = 307.432994 GB avg = 0.027 KB total ukey len = 237.747607 GB avg = 0.015 KB total ukey num = 15.758305163 Billion total entry num = 11.299619111 Billion write speed all = 33.240239289 MB/sec (with seq num) write speed all = 32.067030917 MB/sec (without seq num) 2024/11/24-06:19:23.318063 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294757: 942759 keys, 32349926 bytes, temperature: kWarm 2024/11/24-06:19:23.318134 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400363318097, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294757, "file_size": 32349926, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4475428752, "table_properties": {"data_size": 26165456, "index_size": 1805312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26397252, "raw_average_key_size": 28, "raw_value_size": 107820534, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 942759, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400360, "oldest_key_time": 0, "file_creation_time": 1732400360, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294757, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:19:23.356101 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294758.sst) /dev/shm/ToplingTemp/Topling-287636-9qXB3U 2024/11/24-06:19:24.239654 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294758.sst): first pass time = 0.88's, 151.914'MB/sec 2024/11/24-06:19:25.511093 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294758.sst): old prealloc_size = 147639500, real_size = 27754944 2024/11/24-06:19:25.511686 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294758.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.20 2024/11/24-06:19:25.513656 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294758.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:19:26.363458 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294758.sst): second pass time = 1.27's, 84.853'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 37.293'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3171.495'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.328'MB/sec write SST data time = 0.13's, 243.491'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 23863.412'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 84.853'MB/sec zip pipeline throughput = 84.853'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.20 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.4457 value = 4.1913 dict = 1.00 all = 4.2225 } Zip/UnZip{ index = 0.0957 value = 0.2386 dict = 1.00 all = 0.2368 } ---------------------------- total value len = 2253.877164 GB avg = 0.199 KB (by entry num) total key len = 307.459460 GB avg = 0.027 KB total ukey len = 237.766511 GB avg = 0.015 KB total ukey num = 15.759250359 Billion total entry num = 11.300564307 Billion write speed all = 33.240736084 MB/sec (with seq num) write speed all = 32.067476554 MB/sec (without seq num) 2024/11/24-06:19:26.403015 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294758: 945196 keys, 31786160 bytes, temperature: kWarm 2024/11/24-06:19:26.403705 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400366403050, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294758, "file_size": 31786160, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4355931331, "largest_seqno": 4459404975, "table_properties": {"data_size": 25708720, "index_size": 1809728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400363, "oldest_key_time": 0, "file_creation_time": 1732400363, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294758, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:19:26.441080 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294759.sst) /dev/shm/ToplingTemp/Topling-287636-8OKZrR 2024/11/24-06:19:27.448093 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294759.sst): first pass time = 1.01's, 133.288'MB/sec 2024/11/24-06:19:28.778100 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294759.sst): old prealloc_size = 147639500, real_size = 32700880 2024/11/24-06:19:28.778651 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294759.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923284 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0307 GB avg-val = 117.37 avg-zval = 33.25 2024/11/24-06:19:28.779520 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294759.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:19:29.614691 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294759.sst): second pass time = 1.33's, 81.524'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.977'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3357.349'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.325'MB/sec write SST data time = 0.13's, 290.331'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34188.732'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.524'MB/sec zip pipeline throughput = 81.524'MB/sec entries = 923284 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.37 avg-zval = 33.25 usrkeys = 923284 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.37 avg-zval = 33.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923284 bytes = 3231504 } seq expand size = 6462988 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.4440 value = 3.5296 dict = 1.00 all = 3.6482 } Zip/UnZip{ index = 0.0957 value = 0.2833 dict = 1.00 all = 0.2741 } ---------------------------- total value len = 2253.991993 GB avg = 0.199 KB (by entry num) total key len = 307.485312 GB avg = 0.027 KB total ukey len = 237.784977 GB avg = 0.015 KB total ukey num = 15.760173643 Billion total entry num = 11.301487591 Billion write speed all = 33.241159239 MB/sec (with seq num) write speed all = 32.067853357 MB/sec (without seq num) 2024/11/24-06:19:29.661063 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294759: 923284 keys, 36789960 bytes, temperature: kWarm 2024/11/24-06:19:29.661132 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400369661099, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294759, "file_size": 36789960, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4475432055, "table_properties": {"data_size": 30701824, "index_size": 1768064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25851952, "raw_average_key_size": 28, "raw_value_size": 108365800, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923284, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400366, "oldest_key_time": 0, "file_creation_time": 1732400366, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294759, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:19:29.703487 139957659977280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294760.sst) /dev/shm/ToplingTemp/Topling-287636-5XwHlT 2024/11/24-06:19:29.746769 139957659977280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294760.sst): first pass time = 0.04's, 225.278'MB/sec 2024/11/24-06:19:29.832675 139957659977280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294760.sst): old prealloc_size = 147639500, real_size = 3065056 2024/11/24-06:19:29.832768 139957659977280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294760.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 66718 prefix = 4 raw-key = 0.0011 GB zip-key = 0.0001 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0079 GB zip-val = 0.0029 GB avg-val = 118.00 avg-zval = 43.75 2024/11/24-06:19:29.832862 139957659977280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294760.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0001 GB, waited 0.000 sec, Key+Value = 0.010 GB 2024/11/24-06:19:29.890617 139957659977280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00/294760.sst): second pass time = 0.09's, 92.319'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.06's, 57.504'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1811.228'MB/sec (index lex order gen) rebuild zvType time = 0.05's, 0.346'MB/sec write SST data time = 0.01's, 346.160'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.080'MB, throughput = 2808.739'MB/sec zip my value time = 0.09's, unzip length = 0.008'GB zip my value throughput = 92.319'MB/sec zip pipeline throughput = 92.319'MB/sec entries = 66718 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 43.75 usrkeys = 66718 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 43.75 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 66718 bytes = 200176 } seq expand size = 467026 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0013 GB value = 0.0079 GB dict = 0.08 MB all = 0.0097 GB } __ZipSize{ index = 0.0001 GB value = 0.0029 GB dict = 0.08 MB all = 0.0033 GB } UnZip/Zip{ index = 10.3368 value = 2.6970 dict = 1.00 all = 2.9249 } Zip/UnZip{ index = 0.0967 value = 0.3708 dict = 1.00 all = 0.3419 } ---------------------------- total value len = 2254.000333 GB avg = 0.199 KB (by entry num) total key len = 307.487180 GB avg = 0.027 KB total ukey len = 237.786311 GB avg = 0.015 KB total ukey num = 15.760240361 Billion total entry num = 11.301554309 Billion write speed all = 33.241172681 MB/sec (with seq num) write speed all = 32.067864074 MB/sec (without seq num) 2024/11/24-06:19:29.901420 139957659977280 [compaction_job.cc:1948] [default] [JOB 42731] Generated table #294760: 66718 keys, 3330362 bytes, temperature: kWarm 2024/11/24-06:19:29.901470 139957659977280 EVENT_LOG_v1 {"time_micros": 1732400369901443, "cf_name": "default", "job": 42731, "event": "table_file_creation", "file_number": 294760, "file_size": 3330362, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4385734311, "largest_seqno": 4396548078, "table_properties": {"data_size": 2919120, "index_size": 129088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 1868104, "raw_average_key_size": 28, "raw_value_size": 7872724, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 66718, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400369, "oldest_key_time": 0, "file_creation_time": 1732400369, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294760, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:19:29.913708 139957659977280 [compaction_job.cc:703] job-42731: subcompact[0], size: 0.838454 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42731/att-00 [294736,294737,294738,294739,294740,294741,294742,294743,294744,294745,294746,294747,294748,294749,294750,294751,294752,294753,294754,294755,294756,294757,294758,294759,294760] 2024/11/24-06:19:30.051292 139957659977280 (Original Log Time 2024/11/24-06:18:14.876518) EVENT_LOG_v1 {"time_micros": 1732400294876477, "job": 42731, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [294689], "files_L3": [294709, 294710, 294610, 294611, 294612, 294613, 294614, 294615, 294616, 294617, 294618, 294619, 294620, 294623, 294624, 294625, 294626, 294627, 294628, 294629, 294630, 294631, 294632, 294711], "score": 0.979358, "input_data_size": 860549435, "oldest_snapshot_seqno": 4475862137} 2024/11/24-06:19:30.051295 139957659977280 (Original Log Time 2024/11/24-06:19:30.049673) [compaction_job.cc:2022] [default] [JOB 42731] Compacted 1@2 + 24@3 files to L3 => 838453933 bytes 2024/11/24-06:19:30.051296 139957659977280 (Original Log Time 2024/11/24-06:19:30.051220) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 25 0 0 0] max score 0.03, MB/sec: 11.5 rd, 11.2 wr, level 3, files in(1, 24) out(25 +0 blob) MB in(27.1, 793.6 +0.0 blob) out(799.6 +0.0 blob), read-write-amplify(59.8) write-amplify(29.5) OK, records in: 21680744, records dropped: 101546 output_compression: Snappy 2024/11/24-06:19:30.051300 139957659977280 (Original Log Time 2024/11/24-06:19:30.051265) EVENT_LOG_v1 {"time_micros": 1732400370051243, "cf": "default", "job": 42731, "event": "compaction_finished", "compaction_time_micros": 75037195, "compaction_time_cpu_micros": 32236489, "output_level": 3, "num_output_files": 25, "total_output_size": 838453933, "num_input_records": 21680744, "num_output_records": 21579198, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 576545483, "file_range_sync_nanos": 0, "file_fsync_nanos": 18287, "file_prepare_write_nanos": 66419, "lsm_state": [0, 0, 0, 25, 0, 0, 0]} 2024/11/24-06:19:30.189449 139957659977280 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732400294,"executesMs":75315,"compactionJobId":42731,"attempt":0,"compactionInputRawBytes":3142342424,"compactionInputZipBytes":860549435,"compactionOutputRawBytes":0,"compactionOutputZipBytes":838453933,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-237661f2ec45e4d8e1fe05f685d55cef-cbf30ea713dd322e-00"}}, response = 2024/11/24-06:19:30.189530 139957659977280 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42731/att-00: olev 3, work 75.315 s, result 7.783 ms, install 1.642 ms, input{raw 2.927 GiB zip 820.684 MiB}