2024/11/24-06:23:13.001068 139957458552384 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00: fork to child time = 0.007639 sec 2024/11/24-06:23:13.001891 139957458552384 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00 2024/11/24-06:23:13.009486 139957458552384 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00 2024/11/24-06:23:13.010409 139957458552384 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-06:23:13.010660 139957458552384 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-06:23:13.065272 139957458552384 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42783/att-00/MANIFEST-206347 2024/11/24-06:23:13.090754 139957458552384 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42783/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 294850, last_sequence is 4477391848, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 294653 2024/11/24-06:23:13.090762 139957458552384 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-06:23:13.090898 139957458552384 [version_set.cc:5721] Creating manifest 294850 2024/11/24-06:23:13.123275 139957458552384 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-06:23:13.142706 139957458552384 [compaction_job.cc:2437] [default] [JOB 42783] Compacting 1@2 + 23@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/24-06:23:13.142722 139957458552384 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 32067 Base level 2, inputs: [294785(27MB)], [294822(35MB) 294823(6789KB) 294735(33MB) 294736(8774KB) 294737(33MB) 294738(35MB) 294739(33MB) 294740(35MB) 294741(33MB) 294742(35MB) 294743(33MB) 294744(33MB) 294745(35MB) 294746(33MB) 294747(35MB) 294748(33MB) 294749(35MB) 294750(34MB) 294751(33MB) 294752(35MB) 294753(33MB) 294754(35MB) 294824(30MB)] 2024/11/24-06:23:13.149401 139957458552384 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:23:13.149431 139957458552384 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:23:13.149451 139957458552384 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-06:23:13.149462 139957458552384 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-06:23:13.149473 139957458552384 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-06:23:13.880001 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294851.sst) /dev/shm/ToplingTemp/Topling-288357-9vK5Cq 2024/11/24-06:23:14.423172 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294851.sst): first pass time = 0.54's, 247.121'MB/sec 2024/11/24-06:23:15.764053 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294851.sst): old prealloc_size = 147639500, real_size = 33137696 2024/11/24-06:23:15.764608 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294851.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927890 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.65 avg-zval = 33.56 2024/11/24-06:23:15.765409 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294851.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:16.619750 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294851.sst): second pass time = 1.34's, 80.774'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.520'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3308.019'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 288.910'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34800.142'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.774'MB/sec zip pipeline throughput = 80.774'MB/sec entries = 927890 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.65 avg-zval = 33.56 usrkeys = 927890 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.65 avg-zval = 33.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927890 bytes = 3247632 } seq expand size = 6495230 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5106 value = 3.4758 dict = 1.00 all = 3.6042 } Zip/UnZip{ index = 0.0951 value = 0.2877 dict = 1.00 all = 0.2775 } ---------------------------- total value len = 2251.440305 GB avg = 0.200 KB (by entry num) total key len = 306.908943 GB avg = 0.027 KB total ukey len = 237.373285 GB avg = 0.015 KB total ukey num = 15.739589053 Billion total entry num = 11.280903001 Billion write speed all = 33.103046412 MB/sec (with seq num) write speed all = 31.935317633 MB/sec (without seq num) 2024/11/24-06:23:16.668196 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294851: 927890 keys, 37239276 bytes, temperature: kWarm 2024/11/24-06:23:16.668298 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400596668238, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294851, "file_size": 37239276, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178599, "largest_seqno": 4476940707, "table_properties": {"data_size": 31139920, "index_size": 1765632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25980920, "raw_average_key_size": 28, "raw_value_size": 108236940, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927890, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400593, "oldest_key_time": 0, "file_creation_time": 1732400593, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294851, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:16.705227 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294852.sst) /dev/shm/ToplingTemp/Topling-288357-bCUCcn 2024/11/24-06:23:17.740068 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294852.sst): first pass time = 1.03's, 129.704'MB/sec 2024/11/24-06:23:19.011756 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294852.sst): old prealloc_size = 147639500, real_size = 31470976 2024/11/24-06:23:19.012321 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294852.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.07 2024/11/24-06:23:19.013114 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294852.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:19.826607 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294852.sst): second pass time = 1.27's, 85.357'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.626'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3249.919'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 283.981'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 26247.913'MB/sec zip my value time = 1.27's, unzip length = 0.108'GB zip my value throughput = 85.357'MB/sec zip pipeline throughput = 85.357'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.07 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4466 value = 3.6796 dict = 1.00 all = 3.7757 } Zip/UnZip{ index = 0.0957 value = 0.2718 dict = 1.00 all = 0.2649 } ---------------------------- total value len = 2251.555218 GB avg = 0.200 KB (by entry num) total key len = 306.934684 GB avg = 0.027 KB total ukey len = 237.391671 GB avg = 0.015 KB total ukey num = 15.740508353 Billion total entry num = 11.281822301 Billion write speed all = 33.103492746 MB/sec (with seq num) write speed all = 31.935717263 MB/sec (without seq num) 2024/11/24-06:23:19.871262 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294852: 919300 keys, 35547698 bytes, temperature: kWarm 2024/11/24-06:23:19.871333 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400599871297, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294852, "file_size": 35547698, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4313398651, "largest_seqno": 4464245896, "table_properties": {"data_size": 29480944, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400596, "oldest_key_time": 0, "file_creation_time": 1732400596, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294852, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:19.908333 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294853.sst) /dev/shm/ToplingTemp/Topling-288357-IslrAn 2024/11/24-06:23:20.716472 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294853.sst): first pass time = 0.81's, 69.240'MB/sec 2024/11/24-06:23:21.258926 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294853.sst): old prealloc_size = 147639500, real_size = 13777008 2024/11/24-06:23:21.259222 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294853.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 391852 prefix = 4 raw-key = 0.0063 GB zip-key = 0.0008 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0450 GB zip-val = 0.0129 GB avg-val = 114.79 avg-zval = 32.98 2024/11/24-06:23:21.259578 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294853.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.056 GB 2024/11/24-06:23:21.615349 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294853.sst): second pass time = 0.54's, 83.033'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.36's, 43.246'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2787.543'MB/sec (index lex order gen) rebuild zvType time = 0.30's, 0.324'MB/sec write SST data time = 0.05's, 287.692'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.458'MB, throughput = 14635.371'MB/sec zip my value time = 0.54's, unzip length = 0.045'GB zip my value throughput = 83.033'MB/sec zip pipeline throughput = 83.033'MB/sec entries = 391852 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.79 avg-zval = 32.98 usrkeys = 391852 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.79 avg-zval = 32.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 391852 bytes = 1273536 } seq expand size = 2742964 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0078 GB value = 0.0450 GB dict = 0.46 MB all = 0.0560 GB } __ZipSize{ index = 0.0008 GB value = 0.0129 GB dict = 0.46 MB all = 0.0154 GB } UnZip/Zip{ index = 10.3651 value = 3.4808 dict = 1.00 all = 3.6303 } Zip/UnZip{ index = 0.0965 value = 0.2873 dict = 1.00 all = 0.2755 } ---------------------------- total value len = 2251.602943 GB avg = 0.200 KB (by entry num) total key len = 306.945656 GB avg = 0.027 KB total ukey len = 237.399508 GB avg = 0.015 KB total ukey num = 15.740900205 Billion total entry num = 11.282214153 Billion write speed all = 33.103486056 MB/sec (with seq num) write speed all = 31.935697040 MB/sec (without seq num) 2024/11/24-06:23:21.638651 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294853: 391852 keys, 15412931 bytes, temperature: kWarm 2024/11/24-06:23:21.638712 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400601638679, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294853, "file_size": 15412931, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4431110187, "largest_seqno": 4476940717, "table_properties": {"data_size": 12922784, "index_size": 756096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 10971856, "raw_average_key_size": 28, "raw_value_size": 44981760, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 391852, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400599, "oldest_key_time": 0, "file_creation_time": 1732400599, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294853, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:21.663108 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294854.sst) /dev/shm/ToplingTemp/Topling-288357-tP7b3n 2024/11/24-06:23:22.679494 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294854.sst): first pass time = 1.02's, 132.058'MB/sec 2024/11/24-06:23:24.005756 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294854.sst): old prealloc_size = 147639500, real_size = 31398672 2024/11/24-06:23:24.006309 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294854.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.99 2024/11/24-06:23:24.007119 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294854.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:24.821188 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294854.sst): second pass time = 1.33's, 81.837'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.506'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3325.784'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 281.349'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35877.587'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.837'MB/sec zip pipeline throughput = 81.837'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.99 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6886 dict = 1.00 all = 3.7834 } Zip/UnZip{ index = 0.0957 value = 0.2711 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2251.717855 GB avg = 0.200 KB (by entry num) total key len = 306.971396 GB avg = 0.027 KB total ukey len = 237.417894 GB avg = 0.015 KB total ukey num = 15.741819505 Billion total entry num = 11.283133453 Billion write speed all = 33.103932782 MB/sec (with seq num) write speed all = 31.936097051 MB/sec (without seq num) 2024/11/24-06:23:24.867629 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294854: 919300 keys, 35475378 bytes, temperature: kWarm 2024/11/24-06:23:24.867696 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400604867661, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294854, "file_size": 35475378, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178645, "largest_seqno": 4414357721, "table_properties": {"data_size": 29408512, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400601, "oldest_key_time": 0, "file_creation_time": 1732400601, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294854, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:24.909494 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294855.sst) /dev/shm/ToplingTemp/Topling-288357-2aX8fo 2024/11/24-06:23:25.455044 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294855.sst): first pass time = 0.55's, 246.037'MB/sec 2024/11/24-06:23:26.771666 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294855.sst): old prealloc_size = 147639500, real_size = 32695312 2024/11/24-06:23:26.772273 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294855.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927839 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0307 GB avg-val = 116.66 avg-zval = 33.09 2024/11/24-06:23:26.773080 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294855.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:27.615747 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294855.sst): second pass time = 1.32's, 82.259'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 43.595'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3020.570'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.324'MB/sec write SST data time = 0.13's, 289.812'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35594.807'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.259'MB/sec zip pipeline throughput = 82.259'MB/sec entries = 927839 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 33.09 usrkeys = 927839 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 33.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927839 bytes = 3247456 } seq expand size = 6494873 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5119 value = 3.5259 dict = 1.00 all = 3.6475 } Zip/UnZip{ index = 0.0951 value = 0.2836 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 2251.832588 GB avg = 0.200 KB (by entry num) total key len = 306.997376 GB avg = 0.027 KB total ukey len = 237.436451 GB avg = 0.015 KB total ukey num = 15.742747344 Billion total entry num = 11.284061292 Billion write speed all = 33.104556386 MB/sec (with seq num) write speed all = 31.936666847 MB/sec (without seq num) 2024/11/24-06:23:27.663578 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294855: 927839 keys, 36796784 bytes, temperature: kWarm 2024/11/24-06:23:27.663650 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400607663614, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294855, "file_size": 36796784, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178655, "largest_seqno": 4476940082, "table_properties": {"data_size": 30697856, "index_size": 1765312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979492, "raw_average_key_size": 28, "raw_value_size": 108238254, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927839, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400604, "oldest_key_time": 0, "file_creation_time": 1732400604, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294855, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:27.709144 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294856.sst) /dev/shm/ToplingTemp/Topling-288357-3QQbZp 2024/11/24-06:23:28.704083 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294856.sst): first pass time = 0.99's, 134.907'MB/sec 2024/11/24-06:23:29.999678 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294856.sst): old prealloc_size = 147639500, real_size = 31454896 2024/11/24-06:23:30.000233 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294856.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.05 2024/11/24-06:23:30.001027 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294856.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:30.817780 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294856.sst): second pass time = 1.29's, 83.786'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.433'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3307.448'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 281.943'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34068.387'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.786'MB/sec zip pipeline throughput = 83.786'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.05 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4458 value = 3.6816 dict = 1.00 all = 3.7774 } Zip/UnZip{ index = 0.0957 value = 0.2716 dict = 1.00 all = 0.2647 } ---------------------------- total value len = 2251.947501 GB avg = 0.200 KB (by entry num) total key len = 307.023116 GB avg = 0.027 KB total ukey len = 237.454837 GB avg = 0.015 KB total ukey num = 15.743666644 Billion total entry num = 11.284980592 Billion write speed all = 33.105004660 MB/sec (with seq num) write speed all = 31.937068357 MB/sec (without seq num) 2024/11/24-06:23:30.861024 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294856: 919300 keys, 35531634 bytes, temperature: kWarm 2024/11/24-06:23:30.861093 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400610861057, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294856, "file_size": 35531634, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4300475277, "largest_seqno": 4451975369, "table_properties": {"data_size": 29464736, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400607, "oldest_key_time": 0, "file_creation_time": 1732400607, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294856, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:30.898797 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294857.sst) /dev/shm/ToplingTemp/Topling-288357-Lam6Jn 2024/11/24-06:23:31.834433 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294857.sst): first pass time = 0.94's, 143.456'MB/sec 2024/11/24-06:23:33.175919 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294857.sst): old prealloc_size = 147639500, real_size = 33287488 2024/11/24-06:23:33.176453 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294857.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927513 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.71 avg-zval = 33.74 2024/11/24-06:23:33.177245 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294857.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:34.035087 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294857.sst): second pass time = 1.34's, 80.742'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.518'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3458.636'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 294.485'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32950.415'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.742'MB/sec zip pipeline throughput = 80.742'MB/sec entries = 927513 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.71 avg-zval = 33.74 usrkeys = 927513 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.71 avg-zval = 33.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927513 bytes = 3246304 } seq expand size = 6492591 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5113 value = 3.4594 dict = 1.00 all = 3.5898 } Zip/UnZip{ index = 0.0951 value = 0.2891 dict = 1.00 all = 0.2786 } ---------------------------- total value len = 2252.062241 GB avg = 0.200 KB (by entry num) total key len = 307.049086 GB avg = 0.027 KB total ukey len = 237.473387 GB avg = 0.015 KB total ukey num = 15.744594157 Billion total entry num = 11.285908105 Billion write speed all = 33.105447100 MB/sec (with seq num) write speed all = 31.937463418 MB/sec (without seq num) 2024/11/24-06:23:34.082998 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294857: 927513 keys, 37388238 bytes, temperature: kWarm 2024/11/24-06:23:34.083069 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400614083034, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294857, "file_size": 37388238, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178675, "largest_seqno": 4476940223, "table_properties": {"data_size": 31290608, "index_size": 1764800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25970364, "raw_average_key_size": 28, "raw_value_size": 108247442, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927513, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400610, "oldest_key_time": 0, "file_creation_time": 1732400610, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294857, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:34.122256 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294858.sst) /dev/shm/ToplingTemp/Topling-288357-9Tax6q 2024/11/24-06:23:35.117514 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294858.sst): first pass time = 1.00's, 134.862'MB/sec 2024/11/24-06:23:36.432286 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294858.sst): old prealloc_size = 147639500, real_size = 31214656 2024/11/24-06:23:36.432835 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294858.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921856 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.60 avg-zval = 31.70 2024/11/24-06:23:36.434581 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294858.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:37.246442 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294858.sst): second pass time = 1.31's, 82.540'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.357'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3350.571'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 273.432'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 28568.250'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.540'MB/sec zip pipeline throughput = 82.540'MB/sec entries = 921856 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.60 avg-zval = 31.70 usrkeys = 921856 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.60 avg-zval = 31.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921856 bytes = 3226512 } seq expand size = 6452992 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4441 value = 3.7102 dict = 1.00 all = 3.8024 } Zip/UnZip{ index = 0.0957 value = 0.2695 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 2252.177100 GB avg = 0.200 KB (by entry num) total key len = 307.074898 GB avg = 0.027 KB total ukey len = 237.491824 GB avg = 0.015 KB total ukey num = 15.745516013 Billion total entry num = 11.286829961 Billion write speed all = 33.105891540 MB/sec (with seq num) write speed all = 31.937860978 MB/sec (without seq num) 2024/11/24-06:23:37.289684 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294858: 921856 keys, 35298180 bytes, temperature: kWarm 2024/11/24-06:23:37.289754 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400617289719, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294858, "file_size": 35298180, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4341117826, "largest_seqno": 4476940109, "table_properties": {"data_size": 29218672, "index_size": 1765312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25811968, "raw_average_key_size": 28, "raw_value_size": 108405820, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921856, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400614, "oldest_key_time": 0, "file_creation_time": 1732400614, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294858, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:37.330378 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294859.sst) /dev/shm/ToplingTemp/Topling-288357-14Sxon 2024/11/24-06:23:38.350926 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294859.sst): first pass time = 1.02's, 131.520'MB/sec 2024/11/24-06:23:39.690034 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294859.sst): old prealloc_size = 147639500, real_size = 33017904 2024/11/24-06:23:39.690609 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294859.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925152 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.08 avg-zval = 33.52 2024/11/24-06:23:39.691376 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294859.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:40.540842 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294859.sst): second pass time = 1.34's, 80.934'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.620'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3209.781'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 287.506'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36088.173'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.934'MB/sec zip pipeline throughput = 80.934'MB/sec entries = 925152 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.08 avg-zval = 33.52 usrkeys = 925152 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.08 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925152 bytes = 3238048 } seq expand size = 6476064 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.4443 value = 3.4923 dict = 1.00 all = 3.6167 } Zip/UnZip{ index = 0.0957 value = 0.2863 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 2252.291889 GB avg = 0.200 KB (by entry num) total key len = 307.100803 GB avg = 0.027 KB total ukey len = 237.510327 GB avg = 0.015 KB total ukey num = 15.746441165 Billion total entry num = 11.287755113 Billion write speed all = 33.106300680 MB/sec (with seq num) write speed all = 31.938224156 MB/sec (without seq num) 2024/11/24-06:23:40.586774 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294859: 925152 keys, 37111060 bytes, temperature: kWarm 2024/11/24-06:23:40.586840 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400620586805, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294859, "file_size": 37111060, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178686, "largest_seqno": 4476935624, "table_properties": {"data_size": 31014880, "index_size": 1771584, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25904256, "raw_average_key_size": 28, "raw_value_size": 108313604, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925152, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400617, "oldest_key_time": 0, "file_creation_time": 1732400617, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294859, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:40.624381 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294860.sst) /dev/shm/ToplingTemp/Topling-288357-L89Rkq 2024/11/24-06:23:41.554949 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294860.sst): first pass time = 0.93's, 144.239'MB/sec 2024/11/24-06:23:42.861439 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294860.sst): old prealloc_size = 147639500, real_size = 30951712 2024/11/24-06:23:42.862040 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294860.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927776 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0289 GB avg-val = 116.67 avg-zval = 31.20 2024/11/24-06:23:42.863873 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294860.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:43.684213 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294860.sst): second pass time = 1.31's, 82.903'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.604'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3092.516'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 273.037'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29602.434'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.903'MB/sec zip pipeline throughput = 82.903'MB/sec entries = 927776 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.67 avg-zval = 31.20 usrkeys = 927776 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.67 avg-zval = 31.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927776 bytes = 3247232 } seq expand size = 6494432 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4434 value = 3.7398 dict = 1.00 all = 3.8290 } Zip/UnZip{ index = 0.0958 value = 0.2674 dict = 1.00 all = 0.2612 } ---------------------------- total value len = 2252.406624 GB avg = 0.200 KB (by entry num) total key len = 307.126780 GB avg = 0.027 KB total ukey len = 237.528883 GB avg = 0.015 KB total ukey num = 15.747368941 Billion total entry num = 11.288682889 Billion write speed all = 33.106774694 MB/sec (with seq num) write speed all = 31.938649659 MB/sec (without seq num) 2024/11/24-06:23:43.726513 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294860: 927776 keys, 35052920 bytes, temperature: kWarm 2024/11/24-06:23:43.726583 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400623726547, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294860, "file_size": 35052920, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178700, "largest_seqno": 4476940778, "table_properties": {"data_size": 28942800, "index_size": 1776768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25977728, "raw_average_key_size": 28, "raw_value_size": 108240096, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927776, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400620, "oldest_key_time": 0, "file_creation_time": 1732400620, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294860, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:43.769153 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294861.sst) /dev/shm/ToplingTemp/Topling-288357-4SJWjn 2024/11/24-06:23:44.747157 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294861.sst): first pass time = 0.98's, 137.241'MB/sec 2024/11/24-06:23:46.026032 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294861.sst): old prealloc_size = 147639500, real_size = 31533072 2024/11/24-06:23:46.026572 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.14 2024/11/24-06:23:46.028020 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294861.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:46.849464 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294861.sst): second pass time = 1.28's, 84.908'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.247'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3415.330'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.331'MB/sec write SST data time = 0.13's, 276.987'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32569.190'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.908'MB/sec zip pipeline throughput = 84.908'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.14 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4458 value = 3.6719 dict = 1.00 all = 3.7691 } Zip/UnZip{ index = 0.0957 value = 0.2723 dict = 1.00 all = 0.2653 } ---------------------------- total value len = 2252.521536 GB avg = 0.200 KB (by entry num) total key len = 307.152521 GB avg = 0.027 KB total ukey len = 237.547269 GB avg = 0.015 KB total ukey num = 15.748288241 Billion total entry num = 11.289602189 Billion write speed all = 33.107238536 MB/sec (with seq num) write speed all = 31.939066201 MB/sec (without seq num) 2024/11/24-06:23:46.892670 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294861: 919300 keys, 35609778 bytes, temperature: kWarm 2024/11/24-06:23:46.892741 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400626892706, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294861, "file_size": 35609778, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265722665, "largest_seqno": 4418640153, "table_properties": {"data_size": 29542912, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400623, "oldest_key_time": 0, "file_creation_time": 1732400623, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:46.927423 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294862.sst) /dev/shm/ToplingTemp/Topling-288357-5bWo4q 2024/11/24-06:23:47.960147 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294862.sst): first pass time = 1.03's, 129.970'MB/sec 2024/11/24-06:23:49.330367 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294862.sst): old prealloc_size = 147639500, real_size = 32955584 2024/11/24-06:23:49.330876 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927806 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.66 avg-zval = 33.37 2024/11/24-06:23:49.332918 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294862.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:50.187876 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294862.sst): second pass time = 1.37's, 79.039'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.216'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3754.496'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 288.202'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32036.782'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.039'MB/sec zip pipeline throughput = 79.039'MB/sec entries = 927806 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 33.37 usrkeys = 927806 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 33.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927806 bytes = 3247328 } seq expand size = 6494642 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5123 value = 3.4963 dict = 1.00 all = 3.6219 } Zip/UnZip{ index = 0.0951 value = 0.2860 dict = 1.00 all = 0.2761 } ---------------------------- total value len = 2252.636270 GB avg = 0.200 KB (by entry num) total key len = 307.178499 GB avg = 0.027 KB total ukey len = 237.565825 GB avg = 0.015 KB total ukey num = 15.749216047 Billion total entry num = 11.290529995 Billion write speed all = 33.107628959 MB/sec (with seq num) write speed all = 31.939411064 MB/sec (without seq num) 2024/11/24-06:23:50.233185 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294862: 927806 keys, 37056808 bytes, temperature: kWarm 2024/11/24-06:23:50.233257 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400630233220, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294862, "file_size": 37056808, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178719, "largest_seqno": 4476940140, "table_properties": {"data_size": 30958256, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25978568, "raw_average_key_size": 28, "raw_value_size": 108239268, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927806, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400626, "oldest_key_time": 0, "file_creation_time": 1732400626, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:50.272464 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294863.sst) /dev/shm/ToplingTemp/Topling-288357-hY16Gn 2024/11/24-06:23:51.284875 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294863.sst): first pass time = 1.01's, 132.577'MB/sec 2024/11/24-06:23:52.608273 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294863.sst): old prealloc_size = 147639500, real_size = 31502560 2024/11/24-06:23:52.608843 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294863.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.10 2024/11/24-06:23:52.610843 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294863.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:53.431200 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294863.sst): second pass time = 1.32's, 82.051'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.236'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3227.657'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 274.437'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33839.940'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.051'MB/sec zip pipeline throughput = 82.051'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.10 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.4439 value = 3.6757 dict = 1.00 all = 3.7724 } Zip/UnZip{ index = 0.0957 value = 0.2721 dict = 1.00 all = 0.2651 } ---------------------------- total value len = 2252.751183 GB avg = 0.200 KB (by entry num) total key len = 307.204240 GB avg = 0.027 KB total ukey len = 237.584211 GB avg = 0.015 KB total ukey num = 15.750135347 Billion total entry num = 11.291449295 Billion write speed all = 33.108059296 MB/sec (with seq num) write speed all = 31.939795288 MB/sec (without seq num) 2024/11/24-06:23:53.474914 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294863: 919300 keys, 35579250 bytes, temperature: kWarm 2024/11/24-06:23:53.474994 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400633474949, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294863, "file_size": 35579250, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4305058543, "largest_seqno": 4456632768, "table_properties": {"data_size": 29512080, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400630, "oldest_key_time": 0, "file_creation_time": 1732400630, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294863, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:53.515466 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294864.sst) /dev/shm/ToplingTemp/Topling-288357-NRJKvo 2024/11/24-06:23:54.431104 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294864.sst): first pass time = 0.92's, 146.591'MB/sec 2024/11/24-06:23:55.778329 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294864.sst): old prealloc_size = 147639500, real_size = 33361280 2024/11/24-06:23:55.778918 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294864.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928122 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.61 avg-zval = 33.79 2024/11/24-06:23:55.781015 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294864.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:56.644415 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294864.sst): second pass time = 1.35's, 80.384'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.258'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3125.848'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 286.169'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 30706.461'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.384'MB/sec zip pipeline throughput = 80.384'MB/sec entries = 928122 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.61 avg-zval = 33.79 usrkeys = 928122 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.61 avg-zval = 33.79 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928122 bytes = 3248448 } seq expand size = 6496854 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5124 value = 3.4509 dict = 1.00 all = 3.5826 } Zip/UnZip{ index = 0.0951 value = 0.2898 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 2252.865910 GB avg = 0.200 KB (by entry num) total key len = 307.230227 GB avg = 0.027 KB total ukey len = 237.602773 GB avg = 0.015 KB total ukey num = 15.751063469 Billion total entry num = 11.292377417 Billion write speed all = 33.108503292 MB/sec (with seq num) write speed all = 31.940191807 MB/sec (without seq num) 2024/11/24-06:23:56.689814 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294864: 928122 keys, 37463330 bytes, temperature: kWarm 2024/11/24-06:23:56.689885 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400636689849, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294864, "file_size": 37463330, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4261178730, "largest_seqno": 4476940462, "table_properties": {"data_size": 31363312, "index_size": 1765760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25987416, "raw_average_key_size": 28, "raw_value_size": 108230324, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928122, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400633, "oldest_key_time": 0, "file_creation_time": 1732400633, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294864, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:56.735147 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294865.sst) /dev/shm/ToplingTemp/Topling-288357-BbaTwp 2024/11/24-06:23:57.739698 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294865.sst): first pass time = 1.00's, 133.616'MB/sec 2024/11/24-06:23:59.069945 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294865.sst): old prealloc_size = 147639500, real_size = 31083328 2024/11/24-06:23:59.070527 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294865.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921990 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.57 avg-zval = 31.55 2024/11/24-06:23:59.072523 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294865.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:23:59.881756 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294865.sst): second pass time = 1.33's, 81.538'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.321'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3346.327'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.338'MB/sec write SST data time = 0.13's, 272.211'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33250.583'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.538'MB/sec zip pipeline throughput = 81.538'MB/sec entries = 921990 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.57 avg-zval = 31.55 usrkeys = 921990 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.57 avg-zval = 31.55 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921990 bytes = 3226976 } seq expand size = 6453930 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4464 value = 3.7268 dict = 1.00 all = 3.8166 } Zip/UnZip{ index = 0.0957 value = 0.2683 dict = 1.00 all = 0.2620 } ---------------------------- total value len = 2252.980766 GB avg = 0.199 KB (by entry num) total key len = 307.256043 GB avg = 0.027 KB total ukey len = 237.621213 GB avg = 0.015 KB total ukey num = 15.751985459 Billion total entry num = 11.293299407 Billion write speed all = 33.108936356 MB/sec (with seq num) write speed all = 31.940578398 MB/sec (without seq num) 2024/11/24-06:23:59.925734 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294865: 921990 keys, 35167096 bytes, temperature: kWarm 2024/11/24-06:23:59.925809 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400639925768, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294865, "file_size": 35167096, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4342164741, "largest_seqno": 4476940162, "table_properties": {"data_size": 29087472, "index_size": 1765184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25815720, "raw_average_key_size": 28, "raw_value_size": 108402064, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921990, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400636, "oldest_key_time": 0, "file_creation_time": 1732400636, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294865, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:23:59.961129 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294866.sst) /dev/shm/ToplingTemp/Topling-288357-kNwosp 2024/11/24-06:24:00.905253 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294866.sst): first pass time = 0.94's, 142.168'MB/sec 2024/11/24-06:24:02.269687 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294866.sst): old prealloc_size = 147639500, real_size = 33020000 2024/11/24-06:24:02.270294 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294866.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925013 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 117.10 avg-zval = 33.54 2024/11/24-06:24:02.271116 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294866.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:24:03.122566 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294866.sst): second pass time = 1.36's, 79.436'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.516'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3010.896'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 281.281'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33717.308'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.436'MB/sec zip pipeline throughput = 79.436'MB/sec entries = 925013 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.10 avg-zval = 33.54 usrkeys = 925013 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.10 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925013 bytes = 3237568 } seq expand size = 6475091 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5123 value = 3.4909 dict = 1.00 all = 3.6165 } Zip/UnZip{ index = 0.0951 value = 0.2865 dict = 1.00 all = 0.2765 } ---------------------------- total value len = 2253.095558 GB avg = 0.199 KB (by entry num) total key len = 307.281943 GB avg = 0.027 KB total ukey len = 237.639714 GB avg = 0.015 KB total ukey num = 15.752910472 Billion total entry num = 11.294224420 Billion write speed all = 33.109368177 MB/sec (with seq num) write speed all = 31.940963488 MB/sec (without seq num) 2024/11/24-06:24:03.168205 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294866: 925013 keys, 37112750 bytes, temperature: kWarm 2024/11/24-06:24:03.168276 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400643168241, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294866, "file_size": 37112750, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4476938532, "table_properties": {"data_size": 31028688, "index_size": 1759872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900364, "raw_average_key_size": 28, "raw_value_size": 108317434, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925013, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400639, "oldest_key_time": 0, "file_creation_time": 1732400639, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294866, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:24:03.205079 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294867.sst) /dev/shm/ToplingTemp/Topling-288357-jjpq4o 2024/11/24-06:24:04.223443 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294867.sst): first pass time = 1.02's, 131.802'MB/sec 2024/11/24-06:24:05.519783 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294867.sst): old prealloc_size = 147639500, real_size = 31358272 2024/11/24-06:24:05.520340 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294867.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927823 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0294 GB avg-val = 116.66 avg-zval = 31.65 2024/11/24-06:24:05.521112 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294867.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:24:06.345861 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294867.sst): second pass time = 1.30's, 83.569'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 42.927'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3288.237'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.332'MB/sec write SST data time = 0.13's, 277.512'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34644.855'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.569'MB/sec zip pipeline throughput = 83.569'MB/sec entries = 927823 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 31.65 usrkeys = 927823 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.66 avg-zval = 31.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927823 bytes = 3247392 } seq expand size = 6494761 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.5155 value = 3.6864 dict = 1.00 all = 3.7851 } Zip/UnZip{ index = 0.0951 value = 0.2713 dict = 1.00 all = 0.2642 } ---------------------------- total value len = 2253.210292 GB avg = 0.199 KB (by entry num) total key len = 307.307922 GB avg = 0.027 KB total ukey len = 237.658270 GB avg = 0.015 KB total ukey num = 15.753838295 Billion total entry num = 11.295152243 Billion write speed all = 33.109807716 MB/sec (with seq num) write speed all = 31.941355746 MB/sec (without seq num) 2024/11/24-06:24:06.389779 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294867: 927823 keys, 35459624 bytes, temperature: kWarm 2024/11/24-06:24:06.389850 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400646389813, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294867, "file_size": 35459624, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4476940253, "table_properties": {"data_size": 29361456, "index_size": 1764672, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25979044, "raw_average_key_size": 28, "raw_value_size": 108238730, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927823, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400643, "oldest_key_time": 0, "file_creation_time": 1732400643, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294867, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:24:06.428735 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294868.sst) /dev/shm/ToplingTemp/Topling-288357-Epnkpn 2024/11/24-06:24:07.421997 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294868.sst): first pass time = 0.99's, 135.134'MB/sec 2024/11/24-06:24:08.740314 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294868.sst): old prealloc_size = 147639500, real_size = 31520720 2024/11/24-06:24:08.740862 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294868.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.11 2024/11/24-06:24:08.741625 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294868.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:24:09.557735 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294868.sst): second pass time = 1.32's, 82.331'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.550'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3373.822'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 278.665'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31003.829'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.331'MB/sec zip pipeline throughput = 82.331'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 32.11 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 32.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0356 GB } UnZip/Zip{ index = 10.3768 value = 3.6749 dict = 1.00 all = 3.7704 } Zip/UnZip{ index = 0.0964 value = 0.2721 dict = 1.00 all = 0.2652 } ---------------------------- total value len = 2253.325204 GB avg = 0.199 KB (by entry num) total key len = 307.333663 GB avg = 0.027 KB total ukey len = 237.676656 GB avg = 0.015 KB total ukey num = 15.754757595 Billion total entry num = 11.296071543 Billion write speed all = 33.110251332 MB/sec (with seq num) write speed all = 31.941752794 MB/sec (without seq num) 2024/11/24-06:24:09.602708 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294868: 919300 keys, 35597490 bytes, temperature: kWarm 2024/11/24-06:24:09.602780 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400649602744, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294868, "file_size": 35597490, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4275089855, "largest_seqno": 4427557504, "table_properties": {"data_size": 29518848, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400646, "oldest_key_time": 0, "file_creation_time": 1732400646, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294868, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:24:09.638468 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294869.sst) /dev/shm/ToplingTemp/Topling-288357-N1v44q 2024/11/24-06:24:10.668955 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294869.sst): first pass time = 1.03's, 130.252'MB/sec 2024/11/24-06:24:12.040565 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294869.sst): old prealloc_size = 147639500, real_size = 33174160 2024/11/24-06:24:12.041113 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294869.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927536 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0312 GB avg-val = 116.70 avg-zval = 33.61 2024/11/24-06:24:12.041869 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294869.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:24:12.899170 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294869.sst): second pass time = 1.37's, 78.964'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.415'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3348.849'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 287.921'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25426.003'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.964'MB/sec zip pipeline throughput = 78.964'MB/sec entries = 927536 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.70 avg-zval = 33.61 usrkeys = 927536 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.70 avg-zval = 33.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927536 bytes = 3246384 } seq expand size = 6492752 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5153 value = 3.4719 dict = 1.00 all = 3.6008 } Zip/UnZip{ index = 0.0951 value = 0.2880 dict = 1.00 all = 0.2777 } ---------------------------- total value len = 2253.439944 GB avg = 0.199 KB (by entry num) total key len = 307.359634 GB avg = 0.027 KB total ukey len = 237.695207 GB avg = 0.015 KB total ukey num = 15.755685131 Billion total entry num = 11.296999079 Billion write speed all = 33.110640198 MB/sec (with seq num) write speed all = 31.942096200 MB/sec (without seq num) 2024/11/24-06:24:12.945485 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294869: 927536 keys, 37274926 bytes, temperature: kWarm 2024/11/24-06:24:12.945554 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400652945520, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294869, "file_size": 37274926, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4476940575, "table_properties": {"data_size": 31177920, "index_size": 1764160, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25971008, "raw_average_key_size": 28, "raw_value_size": 108246844, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927536, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400649, "oldest_key_time": 0, "file_creation_time": 1732400649, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294869, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:24:12.990508 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294870.sst) /dev/shm/ToplingTemp/Topling-288357-JKtlVn 2024/11/24-06:24:13.985960 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294870.sst): first pass time = 1.00's, 134.837'MB/sec 2024/11/24-06:24:15.310932 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294870.sst): old prealloc_size = 147639500, real_size = 31460160 2024/11/24-06:24:15.311540 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294870.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 118.00 avg-zval = 32.06 2024/11/24-06:24:15.312310 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294870.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:24:16.126364 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294870.sst): second pass time = 1.32's, 81.924'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.582'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3015.100'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 281.237'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33328.521'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.924'MB/sec zip pipeline throughput = 81.924'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.06 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.06 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4443 value = 3.6810 dict = 1.00 all = 3.7769 } Zip/UnZip{ index = 0.0957 value = 0.2717 dict = 1.00 all = 0.2648 } ---------------------------- total value len = 2253.554856 GB avg = 0.199 KB (by entry num) total key len = 307.385374 GB avg = 0.027 KB total ukey len = 237.713593 GB avg = 0.015 KB total ukey num = 15.756604431 Billion total entry num = 11.297918379 Billion write speed all = 33.111077186 MB/sec (with seq num) write speed all = 31.942486860 MB/sec (without seq num) 2024/11/24-06:24:16.169537 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294870: 919300 keys, 35536882 bytes, temperature: kWarm 2024/11/24-06:24:16.169607 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400656169571, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294870, "file_size": 35536882, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4320158687, "largest_seqno": 4471184070, "table_properties": {"data_size": 29469744, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400652, "oldest_key_time": 0, "file_creation_time": 1732400652, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294870, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:24:16.206503 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294871.sst) /dev/shm/ToplingTemp/Topling-288357-WcrZLn 2024/11/24-06:24:17.241319 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294871.sst): first pass time = 1.03's, 129.707'MB/sec 2024/11/24-06:24:18.599454 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294871.sst): old prealloc_size = 147639500, real_size = 33505744 2024/11/24-06:24:18.600017 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294871.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927920 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0315 GB avg-val = 116.64 avg-zval = 33.94 2024/11/24-06:24:18.600800 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294871.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:24:19.453612 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294871.sst): second pass time = 1.36's, 79.768'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.034'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3317.970'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 295.169'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32435.391'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.768'MB/sec zip pipeline throughput = 79.768'MB/sec entries = 927920 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.64 avg-zval = 33.94 usrkeys = 927920 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.64 avg-zval = 33.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927920 bytes = 3247728 } seq expand size = 6495440 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4458 value = 3.4364 dict = 1.00 all = 3.5686 } Zip/UnZip{ index = 0.0957 value = 0.2910 dict = 1.00 all = 0.2802 } ---------------------------- total value len = 2253.669588 GB avg = 0.199 KB (by entry num) total key len = 307.411356 GB avg = 0.027 KB total ukey len = 237.732151 GB avg = 0.015 KB total ukey num = 15.757532351 Billion total entry num = 11.298846299 Billion write speed all = 33.111472087 MB/sec (with seq num) write speed all = 31.942836055 MB/sec (without seq num) 2024/11/24-06:24:19.498964 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294871: 927920 keys, 37610636 bytes, temperature: kWarm 2024/11/24-06:24:19.499041 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400659499005, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294871, "file_size": 37610636, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4476940537, "table_properties": {"data_size": 31496960, "index_size": 1776640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25981760, "raw_average_key_size": 28, "raw_value_size": 108236028, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927920, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400656, "oldest_key_time": 0, "file_creation_time": 1732400656, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294871, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:24:19.540315 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294872.sst) /dev/shm/ToplingTemp/Topling-288357-Yp2FXm 2024/11/24-06:24:20.440849 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294872.sst): first pass time = 0.90's, 149.050'MB/sec 2024/11/24-06:24:21.702715 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294872.sst): old prealloc_size = 147639500, real_size = 28892800 2024/11/24-06:24:21.703301 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294872.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 939773 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1079 GB zip-val = 0.0269 GB avg-val = 114.82 avg-zval = 28.58 2024/11/24-06:24:21.704077 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294872.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-06:24:22.540058 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294872.sst): second pass time = 1.26's, 85.564'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 39.445'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3215.013'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.331'MB/sec write SST data time = 0.13's, 258.434'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34223.684'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.564'MB/sec zip pipeline throughput = 85.564'MB/sec entries = 939773 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.82 avg-zval = 28.58 usrkeys = 939773 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.82 avg-zval = 28.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 939773 bytes = 3289216 } seq expand size = 6578411 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0269 GB dict = 1.08 MB all = 0.0330 GB } UnZip/Zip{ index = 10.4445 value = 4.0176 dict = 1.00 all = 4.0638 } Zip/UnZip{ index = 0.0957 value = 0.2489 dict = 1.00 all = 0.2461 } ---------------------------- total value len = 2253.784070 GB avg = 0.199 KB (by entry num) total key len = 307.437669 GB avg = 0.027 KB total ukey len = 237.750947 GB avg = 0.015 KB total ukey num = 15.758472124 Billion total entry num = 11.299786072 Billion write speed all = 33.111971106 MB/sec (with seq num) write speed all = 31.943284509 MB/sec (without seq num) 2024/11/24-06:24:22.583477 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294872: 939773 keys, 33027934 bytes, temperature: kWarm 2024/11/24-06:24:22.583549 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400662583512, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294872, "file_size": 33027934, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4476938948, "table_properties": {"data_size": 26858096, "index_size": 1799552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26313644, "raw_average_key_size": 28, "raw_value_size": 107904094, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 939773, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400659, "oldest_key_time": 0, "file_creation_time": 1732400659, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294872, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:24:22.619607 139957458552384 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294873.sst) /dev/shm/ToplingTemp/Topling-288357-vOIx1n 2024/11/24-06:24:22.684530 139957458552384 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294873.sst): first pass time = 0.06's, 235.671'MB/sec 2024/11/24-06:24:22.817577 139957458552384 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294873.sst): old prealloc_size = 147639500, real_size = 4288544 2024/11/24-06:24:22.817694 139957458552384 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294873.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 107676 prefix = 4 raw-key = 0.0017 GB zip-key = 0.0002 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0123 GB zip-val = 0.0041 GB avg-val = 114.00 avg-zval = 37.65 2024/11/24-06:24:22.818202 139957458552384 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294873.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0002 GB, waited 0.000 sec, Key+Value = 0.015 GB 2024/11/24-06:24:22.911906 139957458552384 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00/294873.sst): second pass time = 0.13's, 92.755'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.09's, 49.981'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2253.761'MB/sec (index lex order gen) rebuild zvType time = 0.08's, 0.341'MB/sec write SST data time = 0.02's, 307.217'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.128'MB, throughput = 4796.255'MB/sec zip my value time = 0.13's, unzip length = 0.012'GB zip my value throughput = 92.755'MB/sec zip pipeline throughput = 92.755'MB/sec entries = 107676 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.00 avg-zval = 37.65 usrkeys = 107676 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.00 avg-zval = 37.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 107676 bytes = 323040 } seq expand size = 753732 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0022 GB value = 0.0123 GB dict = 0.13 MB all = 0.0153 GB } __ZipSize{ index = 0.0002 GB value = 0.0041 GB dict = 0.13 MB all = 0.0047 GB } UnZip/Zip{ index = 10.3790 value = 3.0279 dict = 1.00 all = 3.2438 } Zip/UnZip{ index = 0.0963 value = 0.3303 dict = 1.00 all = 0.3083 } ---------------------------- total value len = 2253.797099 GB avg = 0.199 KB (by entry num) total key len = 307.440684 GB avg = 0.027 KB total ukey len = 237.753100 GB avg = 0.015 KB total ukey num = 15.758579800 Billion total entry num = 11.299893748 Billion write speed all = 33.112019340 MB/sec (with seq num) write speed all = 31.943327225 MB/sec (without seq num) 2024/11/24-06:24:22.925366 139957458552384 [compaction_job.cc:1948] [default] [JOB 42783] Generated table #294873: 107676 keys, 4713645 bytes, temperature: kWarm 2024/11/24-06:24:22.925425 139957458552384 EVENT_LOG_v1 {"time_micros": 1732400662925397, "cf_name": "default", "job": 42783, "event": "table_file_creation", "file_number": 294873, "file_size": 4713645, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4344480250, "largest_seqno": 4355793534, "table_properties": {"data_size": 4053968, "index_size": 207488, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3014928, "raw_average_key_size": 28, "raw_value_size": 12275064, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 107676, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732400662, "oldest_key_time": 0, "file_creation_time": 1732400662, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 294873, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-06:24:22.936144 139957458552384 [compaction_job.cc:703] job-42783: subcompact[0], size: 0.778064 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-42783/att-00 [294851,294852,294853,294854,294855,294856,294857,294858,294859,294860,294861,294862,294863,294864,294865,294866,294867,294868,294869,294870,294871,294872,294873] 2024/11/24-06:24:23.063091 139957458552384 (Original Log Time 2024/11/24-06:23:13.142778) EVENT_LOG_v1 {"time_micros": 1732400593142737, "job": 42783, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [294785], "files_L3": [294822, 294823, 294735, 294736, 294737, 294738, 294739, 294740, 294741, 294742, 294743, 294744, 294745, 294746, 294747, 294748, 294749, 294750, 294751, 294752, 294753, 294754, 294824], "score": 0.988614, "input_data_size": 801847760, "oldest_snapshot_seqno": 4477391781} 2024/11/24-06:24:23.063094 139957458552384 (Original Log Time 2024/11/24-06:24:23.061499) [compaction_job.cc:2022] [default] [JOB 42783] Compacted 1@2 + 23@3 files to L3 => 778064248 bytes 2024/11/24-06:24:23.063096 139957458552384 (Original Log Time 2024/11/24-06:24:23.063012) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 11.5 rd, 11.1 wr, level 3, files in(1, 23) out(23 +0 blob) MB in(27.1, 737.6 +0.0 blob) out(742.0 +0.0 blob), read-write-amplify(55.6) write-amplify(27.4) OK, records in: 20024962, records dropped: 106325 output_compression: Snappy 2024/11/24-06:24:23.063099 139957458552384 (Original Log Time 2024/11/24-06:24:23.063066) EVENT_LOG_v1 {"time_micros": 1732400663063040, "cf": "default", "job": 42783, "event": "compaction_finished", "compaction_time_micros": 69793370, "compaction_time_cpu_micros": 29842378, "output_level": 3, "num_output_files": 23, "total_output_size": 778064248, "num_input_records": 20024962, "num_output_records": 19918637, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 531205093, "file_range_sync_nanos": 0, "file_fsync_nanos": 16429, "file_prepare_write_nanos": 57382, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/24-06:24:23.203285 139957458552384 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732400592,"executesMs":70054,"compactionJobId":42783,"attempt":0,"compactionInputRawBytes":2904915552,"compactionInputZipBytes":801847760,"compactionOutputRawBytes":0,"compactionOutputZipBytes":778064248,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-c5c1e51883eed02eabf1060144211546-f930adbd0305e76d-00"}}, response = 2024/11/24-06:24:23.203356 139957458552384 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-42783/att-00: olev 3, work 70.054 s, result 5.988 ms, install 1.616 ms, input{raw 2.705 GiB zip 764.702 MiB}