2024/11/24-07:06:23.154364 139957626406464 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00: fork to child time = 0.009503 sec 2024/11/24-07:06:23.155180 139957626406464 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00 2024/11/24-07:06:23.161233 139957626406464 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00 2024/11/24-07:06:23.162149 139957626406464 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-07:06:23.162432 139957626406464 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-07:06:23.216231 139957626406464 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-43189/att-00/MANIFEST-206347 2024/11/24-07:06:23.232284 139957626406464 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-43189/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 295807, last_sequence is 4490449420, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 295693 2024/11/24-07:06:23.232292 139957626406464 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-07:06:23.232444 139957626406464 [version_set.cc:5721] Creating manifest 295807 2024/11/24-07:06:23.267857 139957626406464 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-07:06:23.288781 139957626406464 [compaction_job.cc:2437] [default] [JOB 43189] Compacting 1@2 + 27@3 files to L3, score 0.93, subcompactions 1 : 1 2024/11/24-07:06:23.288807 139957626406464 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 32372 Base level 2, inputs: [295802(27MB)], [295743(35MB) 295752(31MB) 295753(30MB) 295754(35MB) 295755(33MB) 295756(30MB) 295757(35MB) 295758(33MB) 295759(33MB) 295760(33MB) 295761(33MB) 295762(35MB) 295763(33MB) 295764(33MB) 295765(33MB) 295766(33MB) 295767(35MB) 295768(33MB) 295769(32MB) 295770(33MB) 295771(33MB) 295772(35MB) 295773(33MB) 295774(32MB) 295775(34MB) 295776(33MB) 295777(20MB)] 2024/11/24-07:06:23.296889 139957626406464 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-07:06:23.296921 139957626406464 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-07:06:23.296942 139957626406464 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-07:06:23.296947 139957626406464 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-07:06:23.296958 139957626406464 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-07:06:23.882728 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295808.sst) /dev/shm/ToplingTemp/Topling-294374-1Rhv8o 2024/11/24-07:06:24.464253 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295808.sst): first pass time = 0.58's, 230.818'MB/sec 2024/11/24-07:06:26.447989 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295808.sst): old prealloc_size = 147639500, real_size = 33372432 2024/11/24-07:06:26.448707 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295808.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928335 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0314 GB avg-val = 116.58 avg-zval = 33.78 2024/11/24-07:06:26.449628 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295808.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:06:27.371804 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295808.sst): second pass time = 1.97's, 54.960'MB/sec, value only(80.6% of KV) wait indexing time = 0.01's, remap KeyValue time = 0.92's, 40.568'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2575.747'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.16's, 229.349'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31364.387'MB/sec zip my value time = 1.97's, unzip length = 0.108'GB zip my value throughput = 54.960'MB/sec zip pipeline throughput = 85.202'MB/sec entries = 928335 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.58 avg-zval = 33.78 usrkeys = 928335 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.58 avg-zval = 33.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928335 bytes = 3249184 } seq expand size = 6498345 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4452 value = 3.4507 dict = 1.00 all = 3.5812 } Zip/UnZip{ index = 0.0957 value = 0.2898 dict = 1.00 all = 0.2792 } ---------------------------- total value len = 2251.440296 GB avg = 0.200 KB (by entry num) total key len = 306.908956 GB avg = 0.027 KB total ukey len = 237.373294 GB avg = 0.015 KB total ukey num = 15.739589498 Billion total entry num = 11.280903446 Billion write speed all = 32.029348584 MB/sec (with seq num) write speed all = 30.899495063 MB/sec (without seq num) 2024/11/24-07:06:27.420368 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295808: 928335 keys, 37478430 bytes, temperature: kWarm 2024/11/24-07:06:27.420486 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403187420405, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295808, "file_size": 37478430, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4490438851, "table_properties": {"data_size": 31362624, "index_size": 1777536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25993380, "raw_average_key_size": 28, "raw_value_size": 108224462, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928335, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403183, "oldest_key_time": 0, "file_creation_time": 1732403183, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295808, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:06:27.459928 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295809.sst) /dev/shm/ToplingTemp/Topling-294374-z79pOn 2024/11/24-07:06:29.386311 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295809.sst): first pass time = 1.93's, 69.675'MB/sec 2024/11/24-07:06:32.455274 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295809.sst): old prealloc_size = 147639500, real_size = 29107120 2024/11/24-07:06:32.456029 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295809.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 938412 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1079 GB zip-val = 0.0271 GB avg-val = 115.03 avg-zval = 28.85 2024/11/24-07:06:32.456881 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295809.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:06:33.617075 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295809.sst): second pass time = 3.06's, 35.330'MB/sec, value only(80.4% of KV) wait indexing time = 0.01's, remap KeyValue time = 1.16's, 28.610'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2513.393'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.261'MB/sec write SST data time = 0.26's, 127.496'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 34645.462'MB/sec zip my value time = 3.06's, unzip length = 0.108'GB zip my value throughput = 35.330'MB/sec zip pipeline throughput = 116.686'MB/sec entries = 938412 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.03 avg-zval = 28.85 usrkeys = 938412 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.03 avg-zval = 28.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 938412 bytes = 3284464 } seq expand size = 6568884 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0271 GB dict = 1.08 MB all = 0.0332 GB } UnZip/Zip{ index = 10.4431 value = 3.9868 dict = 1.00 all = 4.0380 } Zip/UnZip{ index = 0.0958 value = 0.2508 dict = 1.00 all = 0.2476 } ---------------------------- total value len = 2251.554807 GB avg = 0.200 KB (by entry num) total key len = 306.935231 GB avg = 0.027 KB total ukey len = 237.392062 GB avg = 0.015 KB total ukey num = 15.740527910 Billion total entry num = 11.281841858 Billion write speed all = 32.028606927 MB/sec (with seq num) write speed all = 30.898747759 MB/sec (without seq num) 2024/11/24-07:06:33.662418 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295809: 938412 keys, 33238306 bytes, temperature: kWarm 2024/11/24-07:06:33.662873 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403193662457, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295809, "file_size": 33238306, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4490437549, "table_properties": {"data_size": 27075168, "index_size": 1797184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26275536, "raw_average_key_size": 28, "raw_value_size": 107942192, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 938412, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403187, "oldest_key_time": 0, "file_creation_time": 1732403187, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295809, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:06:33.744176 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295810.sst) /dev/shm/ToplingTemp/Topling-294374-nVMZbo 2024/11/24-07:06:34.625899 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295810.sst): first pass time = 0.88's, 152.230'MB/sec 2024/11/24-07:06:36.168488 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295810.sst): old prealloc_size = 147639500, real_size = 27620528 2024/11/24-07:06:36.169077 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295810.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.94 raw-val = 0.1078 GB zip-val = 0.0256 GB avg-val = 114.00 avg-zval = 27.03 2024/11/24-07:06:36.169838 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295810.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:06:37.005364 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295810.sst): second pass time = 1.54's, 69.888'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 37.823'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3259.294'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.335'MB/sec write SST data time = 0.13's, 242.339'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 28170.901'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 69.888'MB/sec zip pipeline throughput = 123.419'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.94 avg-val = 114.00 avg-zval = 27.03 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.94 avg-val = 121.00 avg-zval = 27.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0256 GB dict = 1.07 MB all = 0.0317 GB } UnZip/Zip{ index = 10.3119 value = 4.2172 dict = 1.00 all = 4.2404 } Zip/UnZip{ index = 0.0970 value = 0.2371 dict = 1.00 all = 0.2358 } ---------------------------- total value len = 2251.669176 GB avg = 0.200 KB (by entry num) total key len = 306.961697 GB avg = 0.027 KB total ukey len = 237.410966 GB avg = 0.015 KB total ukey num = 15.741473106 Billion total entry num = 11.282787054 Billion write speed all = 32.029011405 MB/sec (with seq num) write speed all = 30.899105504 MB/sec (without seq num) 2024/11/24-07:06:37.046018 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295810: 945196 keys, 31651824 bytes, temperature: kWarm 2024/11/24-07:06:37.046077 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403197046044, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295810, "file_size": 31651824, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4339093113, "largest_seqno": 4441531141, "table_properties": {"data_size": 25550816, "index_size": 1833216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403193, "oldest_key_time": 0, "file_creation_time": 1732403193, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295810, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:06:37.088893 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295811.sst) /dev/shm/ToplingTemp/Topling-294374-rFZz7o 2024/11/24-07:06:37.949427 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295811.sst): first pass time = 0.86's, 155.978'MB/sec 2024/11/24-07:06:39.615274 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295811.sst): old prealloc_size = 147639500, real_size = 32849888 2024/11/24-07:06:39.615810 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295811.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931263 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.12 avg-zval = 33.12 2024/11/24-07:06:39.616591 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295811.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:06:40.495041 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295811.sst): second pass time = 1.67's, 64.946'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 42.014'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3426.872'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.14's, 260.987'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28837.127'MB/sec zip my value time = 1.67's, unzip length = 0.108'GB zip my value throughput = 64.946'MB/sec zip pipeline throughput = 124.859'MB/sec entries = 931263 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.12 avg-zval = 33.12 usrkeys = 931263 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.12 avg-zval = 33.12 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931263 bytes = 3259440 } seq expand size = 6518841 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5118 value = 3.5060 dict = 1.00 all = 3.6314 } Zip/UnZip{ index = 0.0951 value = 0.2852 dict = 1.00 all = 0.2754 } ---------------------------- total value len = 2251.783837 GB avg = 0.200 KB (by entry num) total key len = 306.987772 GB avg = 0.027 KB total ukey len = 237.429591 GB avg = 0.015 KB total ukey num = 15.742404369 Billion total entry num = 11.283718317 Billion write speed all = 32.029373990 MB/sec (with seq num) write speed all = 30.899424189 MB/sec (without seq num) 2024/11/24-07:06:40.540712 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295811: 931263 keys, 36960396 bytes, temperature: kWarm 2024/11/24-07:06:40.540771 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403200540740, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295811, "file_size": 36960396, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4490436915, "table_properties": {"data_size": 30845072, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26075364, "raw_average_key_size": 28, "raw_value_size": 108142506, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931263, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403197, "oldest_key_time": 0, "file_creation_time": 1732403197, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295811, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:06:40.582886 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295812.sst) /dev/shm/ToplingTemp/Topling-294374-H49yQm 2024/11/24-07:06:41.450452 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295812.sst): first pass time = 0.87's, 154.712'MB/sec 2024/11/24-07:06:43.057244 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295812.sst): old prealloc_size = 147639500, real_size = 31464384 2024/11/24-07:06:43.057782 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295812.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919766 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.93 avg-zval = 32.03 2024/11/24-07:06:43.058559 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295812.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:06:43.882546 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295812.sst): second pass time = 1.61's, 67.550'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 43.068'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3430.712'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.331'MB/sec write SST data time = 0.13's, 271.459'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 30252.879'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 67.550'MB/sec zip pipeline throughput = 118.414'MB/sec entries = 919766 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.93 avg-zval = 32.03 usrkeys = 919766 avg-key = 20.00 avg-zkey = 1.93 avg-val = 124.93 avg-zval = 32.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919766 bytes = 3219200 } seq expand size = 6438362 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3764 value = 3.6816 dict = 1.00 all = 3.7763 } Zip/UnZip{ index = 0.0964 value = 0.2716 dict = 1.00 all = 0.2648 } ---------------------------- total value len = 2251.898740 GB avg = 0.200 KB (by entry num) total key len = 307.013526 GB avg = 0.027 KB total ukey len = 237.447987 GB avg = 0.015 KB total ukey num = 15.743324135 Billion total entry num = 11.284638083 Billion write speed all = 32.029776489 MB/sec (with seq num) write speed all = 30.899782498 MB/sec (without seq num) 2024/11/24-07:06:43.932946 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295812: 919766 keys, 35542518 bytes, temperature: kWarm 2024/11/24-07:06:43.933020 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403203932986, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295812, "file_size": 35542518, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4340652921, "largest_seqno": 4490438528, "table_properties": {"data_size": 29461488, "index_size": 1772800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25753448, "raw_average_key_size": 28, "raw_value_size": 108464392, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919766, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403200, "oldest_key_time": 0, "file_creation_time": 1732403200, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295812, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:06:43.970304 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295813.sst) /dev/shm/ToplingTemp/Topling-294374-DxlzZo 2024/11/24-07:06:44.930452 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295813.sst): first pass time = 0.96's, 139.793'MB/sec 2024/11/24-07:06:46.175956 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295813.sst): old prealloc_size = 147639500, real_size = 27597504 2024/11/24-07:06:46.176561 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295813.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1078 GB zip-val = 0.0256 GB avg-val = 114.00 avg-zval = 27.03 2024/11/24-07:06:46.177331 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295813.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:06:47.049196 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295813.sst): second pass time = 1.24's, 86.592'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 36.221'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3146.859'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.321'MB/sec write SST data time = 0.14's, 232.957'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 35406.670'MB/sec zip my value time = 1.24's, unzip length = 0.108'GB zip my value throughput = 86.592'MB/sec zip pipeline throughput = 86.592'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.00 avg-zval = 27.03 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.00 avg-zval = 27.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0256 GB dict = 1.07 MB all = 0.0316 GB } UnZip/Zip{ index = 10.4439 value = 4.2172 dict = 1.00 all = 4.2435 } Zip/UnZip{ index = 0.0957 value = 0.2371 dict = 1.00 all = 0.2357 } ---------------------------- total value len = 2252.013109 GB avg = 0.200 KB (by entry num) total key len = 307.039991 GB avg = 0.027 KB total ukey len = 237.466891 GB avg = 0.015 KB total ukey num = 15.744269331 Billion total entry num = 11.285583279 Billion write speed all = 32.030269723 MB/sec (with seq num) write speed all = 30.900225875 MB/sec (without seq num) 2024/11/24-07:06:47.090742 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295813: 945196 keys, 31628720 bytes, temperature: kWarm 2024/11/24-07:06:47.090811 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403207090776, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295813, "file_size": 31628720, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4357315230, "largest_seqno": 4460727925, "table_properties": {"data_size": 25550960, "index_size": 1810048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403203, "oldest_key_time": 0, "file_creation_time": 1732403203, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295813, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:06:47.130175 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295814.sst) /dev/shm/ToplingTemp/Topling-294374-T804Lm 2024/11/24-07:06:48.316950 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295814.sst): first pass time = 1.19's, 113.098'MB/sec 2024/11/24-07:06:49.963092 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295814.sst): old prealloc_size = 147639500, real_size = 32992080 2024/11/24-07:06:49.963776 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295814.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926702 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.83 avg-zval = 33.45 2024/11/24-07:06:49.964641 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295814.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:06:50.838031 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295814.sst): second pass time = 1.65's, 65.806'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 42.393'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2672.825'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 280.816'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36453.662'MB/sec zip my value time = 1.65's, unzip length = 0.108'GB zip my value throughput = 65.806'MB/sec zip pipeline throughput = 65.806'MB/sec entries = 926702 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 33.45 usrkeys = 926702 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.83 avg-zval = 33.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926702 bytes = 3243472 } seq expand size = 6486914 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5150 value = 3.4929 dict = 1.00 all = 3.6187 } Zip/UnZip{ index = 0.0951 value = 0.2863 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 2252.127866 GB avg = 0.200 KB (by entry num) total key len = 307.065939 GB avg = 0.027 KB total ukey len = 237.485425 GB avg = 0.015 KB total ukey num = 15.745196033 Billion total entry num = 11.286509981 Billion write speed all = 32.030511873 MB/sec (with seq num) write speed all = 30.900428825 MB/sec (without seq num) 2024/11/24-07:06:50.883861 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295814: 926702 keys, 37089802 bytes, temperature: kWarm 2024/11/24-07:06:50.883928 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403210883894, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295814, "file_size": 37089802, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4490439071, "table_properties": {"data_size": 30997568, "index_size": 1762624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947656, "raw_average_key_size": 28, "raw_value_size": 108270116, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926702, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403207, "oldest_key_time": 0, "file_creation_time": 1732403207, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295814, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:06:50.923029 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295815.sst) /dev/shm/ToplingTemp/Topling-294374-bGOXOm 2024/11/24-07:06:52.159742 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295815.sst): first pass time = 1.24's, 108.531'MB/sec 2024/11/24-07:06:53.497232 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295815.sst): old prealloc_size = 147639500, real_size = 31336096 2024/11/24-07:06:53.497799 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295815.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.92 2024/11/24-07:06:53.498543 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295815.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:06:54.310923 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295815.sst): second pass time = 1.33's, 81.410'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.381'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3246.867'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 282.132'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35512.388'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.410'MB/sec zip pipeline throughput = 81.410'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.92 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4439 value = 3.6965 dict = 1.00 all = 3.8024 } Zip/UnZip{ index = 0.0957 value = 0.2705 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 2252.242778 GB avg = 0.200 KB (by entry num) total key len = 307.091679 GB avg = 0.027 KB total ukey len = 237.503811 GB avg = 0.015 KB total ukey num = 15.746115333 Billion total entry num = 11.287429281 Billion write speed all = 32.030879999 MB/sec (with seq num) write speed all = 30.900754027 MB/sec (without seq num) 2024/11/24-07:06:54.356124 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295815: 919300 keys, 35297906 bytes, temperature: kWarm 2024/11/24-07:06:54.356193 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403214356158, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295815, "file_size": 35297906, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4347409080, "largest_seqno": 4468056993, "table_properties": {"data_size": 29345616, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403210, "oldest_key_time": 0, "file_creation_time": 1732403210, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295815, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:06:54.395888 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295816.sst) /dev/shm/ToplingTemp/Topling-294374-I6DA2l 2024/11/24-07:06:55.403601 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295816.sst): first pass time = 1.01's, 133.195'MB/sec 2024/11/24-07:06:56.717294 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295816.sst): old prealloc_size = 147639500, real_size = 30115824 2024/11/24-07:06:56.717868 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295816.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 938142 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1079 GB zip-val = 0.0281 GB avg-val = 115.07 avg-zval = 29.95 2024/11/24-07:06:56.718657 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295816.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:06:57.549066 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295816.sst): second pass time = 1.31's, 82.224'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.176'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3233.274'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.334'MB/sec write SST data time = 0.13's, 266.821'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 36265.697'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.224'MB/sec zip pipeline throughput = 82.224'MB/sec entries = 938142 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.07 avg-zval = 29.95 usrkeys = 938142 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.07 avg-zval = 29.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 938142 bytes = 3283504 } seq expand size = 6566994 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0281 GB dict = 1.08 MB all = 0.0342 GB } UnZip/Zip{ index = 10.5124 value = 3.8421 dict = 1.00 all = 3.9190 } Zip/UnZip{ index = 0.0951 value = 0.2603 dict = 1.00 all = 0.2552 } ---------------------------- total value len = 2252.357295 GB avg = 0.200 KB (by entry num) total key len = 307.117947 GB avg = 0.027 KB total ukey len = 237.522574 GB avg = 0.015 KB total ukey num = 15.747053475 Billion total entry num = 11.288367423 Billion write speed all = 32.031343854 MB/sec (with seq num) write speed all = 30.901169754 MB/sec (without seq num) 2024/11/24-07:06:57.591174 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295816: 938142 keys, 34247904 bytes, temperature: kWarm 2024/11/24-07:06:57.591244 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403217591209, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295816, "file_size": 34247904, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4490438542, "table_properties": {"data_size": 28096288, "index_size": 1784832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26267976, "raw_average_key_size": 28, "raw_value_size": 107949892, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 938142, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403214, "oldest_key_time": 0, "file_creation_time": 1732403214, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295816, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:06:57.627626 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295817.sst) /dev/shm/ToplingTemp/Topling-294374-39rVYn 2024/11/24-07:06:58.628440 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295817.sst): first pass time = 1.00's, 134.114'MB/sec 2024/11/24-07:06:59.919111 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295817.sst): old prealloc_size = 147639500, real_size = 31336096 2024/11/24-07:06:59.919682 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295817.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.92 2024/11/24-07:06:59.920408 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295817.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:00.727656 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295817.sst): second pass time = 1.29's, 84.097'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.658'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3206.225'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 279.347'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34172.467'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.097'MB/sec zip pipeline throughput = 84.097'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.92 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4443 value = 3.6965 dict = 1.00 all = 3.8024 } Zip/UnZip{ index = 0.0957 value = 0.2705 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 2252.472208 GB avg = 0.200 KB (by entry num) total key len = 307.143688 GB avg = 0.027 KB total ukey len = 237.540960 GB avg = 0.015 KB total ukey num = 15.747972775 Billion total entry num = 11.289286723 Billion write speed all = 32.031829889 MB/sec (with seq num) write speed all = 30.901608710 MB/sec (without seq num) 2024/11/24-07:07:00.773469 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295817: 919300 keys, 35297906 bytes, temperature: kWarm 2024/11/24-07:07:00.773538 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403220773503, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295817, "file_size": 35297906, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4275788000, "largest_seqno": 4385584961, "table_properties": {"data_size": 29345680, "index_size": 1760384, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403217, "oldest_key_time": 0, "file_creation_time": 1732403217, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295817, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:00.810963 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295818.sst) /dev/shm/ToplingTemp/Topling-294374-iQdoul 2024/11/24-07:07:01.808933 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295818.sst): first pass time = 1.00's, 134.498'MB/sec 2024/11/24-07:07:03.120418 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295818.sst): old prealloc_size = 147639500, real_size = 31134368 2024/11/24-07:07:03.120960 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295818.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921367 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0291 GB avg-val = 117.67 avg-zval = 31.63 2024/11/24-07:07:03.121705 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295818.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:03.932262 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295818.sst): second pass time = 1.31's, 82.765'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.238'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3481.553'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 273.608'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32185.900'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.765'MB/sec zip pipeline throughput = 82.765'MB/sec entries = 921367 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.67 avg-zval = 31.63 usrkeys = 921367 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.67 avg-zval = 31.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921367 bytes = 3109632 } seq expand size = 6449569 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0291 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4458 value = 3.7207 dict = 1.00 all = 3.8237 } Zip/UnZip{ index = 0.0957 value = 0.2688 dict = 1.00 all = 0.2615 } ---------------------------- total value len = 2252.587077 GB avg = 0.200 KB (by entry num) total key len = 307.169486 GB avg = 0.027 KB total ukey len = 237.559387 GB avg = 0.015 KB total ukey num = 15.748894142 Billion total entry num = 11.290208090 Billion write speed all = 32.032305638 MB/sec (with seq num) write speed all = 30.902037545 MB/sec (without seq num) 2024/11/24-07:07:03.978218 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295818: 921367 keys, 35101444 bytes, temperature: kWarm 2024/11/24-07:07:03.978289 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403223978251, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295818, "file_size": 35101444, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4385584962, "largest_seqno": 4490438552, "table_properties": {"data_size": 29139728, "index_size": 1764096, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25798276, "raw_average_key_size": 28, "raw_value_size": 108419482, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921367, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403220, "oldest_key_time": 0, "file_creation_time": 1732403220, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295818, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:04.021652 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295819.sst) /dev/shm/ToplingTemp/Topling-294374-pj8Etl 2024/11/24-07:07:05.052551 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295819.sst): first pass time = 1.03's, 130.200'MB/sec 2024/11/24-07:07:06.378786 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295819.sst): old prealloc_size = 147639500, real_size = 32776752 2024/11/24-07:07:06.379310 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295819.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931608 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0308 GB avg-val = 116.07 avg-zval = 33.03 2024/11/24-07:07:06.380083 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295819.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:07.236265 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295819.sst): second pass time = 1.33's, 81.601'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.021'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3525.281'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 292.040'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 33408.785'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.601'MB/sec zip pipeline throughput = 81.601'MB/sec entries = 931608 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.07 avg-zval = 33.03 usrkeys = 931608 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.07 avg-zval = 33.03 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931608 bytes = 3260640 } seq expand size = 6521256 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5123 value = 3.5141 dict = 1.00 all = 3.6385 } Zip/UnZip{ index = 0.0951 value = 0.2846 dict = 1.00 all = 0.2748 } ---------------------------- total value len = 2252.701731 GB avg = 0.200 KB (by entry num) total key len = 307.195571 GB avg = 0.027 KB total ukey len = 237.578019 GB avg = 0.015 KB total ukey num = 15.749825750 Billion total entry num = 11.291139698 Billion write speed all = 32.032742405 MB/sec (with seq num) write speed all = 30.902427781 MB/sec (without seq num) 2024/11/24-07:07:07.281855 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295819: 931608 keys, 36888054 bytes, temperature: kWarm 2024/11/24-07:07:07.281925 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403227281890, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295819, "file_size": 36888054, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4490432752, "table_properties": {"data_size": 30771232, "index_size": 1772416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26085024, "raw_average_key_size": 28, "raw_value_size": 108132716, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931608, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403224, "oldest_key_time": 0, "file_creation_time": 1732403224, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295819, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:07.318097 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295820.sst) /dev/shm/ToplingTemp/Topling-294374-fYmyLp 2024/11/24-07:07:08.313480 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295820.sst): first pass time = 1.00's, 134.846'MB/sec 2024/11/24-07:07:09.609728 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295820.sst): old prealloc_size = 147639500, real_size = 31424416 2024/11/24-07:07:09.610282 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295820.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.02 2024/11/24-07:07:09.611033 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295820.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:10.426056 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295820.sst): second pass time = 1.30's, 83.740'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.349'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3335.243'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 280.781'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 32353.590'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.740'MB/sec zip pipeline throughput = 83.740'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.02 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4458 value = 3.6854 dict = 1.00 all = 3.7929 } Zip/UnZip{ index = 0.0957 value = 0.2713 dict = 1.00 all = 0.2636 } ---------------------------- total value len = 2252.816643 GB avg = 0.200 KB (by entry num) total key len = 307.221311 GB avg = 0.027 KB total ukey len = 237.596405 GB avg = 0.015 KB total ukey num = 15.750745050 Billion total entry num = 11.292058998 Billion write speed all = 32.033223835 MB/sec (with seq num) write speed all = 30.902862301 MB/sec (without seq num) 2024/11/24-07:07:10.469297 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295820: 919300 keys, 35386226 bytes, temperature: kWarm 2024/11/24-07:07:10.469371 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403230469334, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295820, "file_size": 35386226, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4321388618, "largest_seqno": 4432385300, "table_properties": {"data_size": 29434256, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403227, "oldest_key_time": 0, "file_creation_time": 1732403227, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295820, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:10.505794 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295821.sst) /dev/shm/ToplingTemp/Topling-294374-TFr8qm 2024/11/24-07:07:11.501960 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295821.sst): first pass time = 1.00's, 134.740'MB/sec 2024/11/24-07:07:12.803921 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295821.sst): old prealloc_size = 147639500, real_size = 30044304 2024/11/24-07:07:12.804538 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295821.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931925 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0280 GB avg-val = 116.02 avg-zval = 30.07 2024/11/24-07:07:12.805281 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295821.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:13.623456 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295821.sst): second pass time = 1.30's, 83.131'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.394'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2933.963'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.13's, 269.309'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30249.986'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.131'MB/sec zip pipeline throughput = 83.131'MB/sec entries = 931925 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.02 avg-zval = 30.07 usrkeys = 931925 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.02 avg-zval = 30.07 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931925 bytes = 3028768 } seq expand size = 6523475 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0280 GB dict = 1.08 MB all = 0.0339 GB } UnZip/Zip{ index = 10.4450 value = 3.8579 dict = 1.00 all = 3.9567 } Zip/UnZip{ index = 0.0957 value = 0.2592 dict = 1.00 all = 0.2527 } ---------------------------- total value len = 2252.931290 GB avg = 0.199 KB (by entry num) total key len = 307.247405 GB avg = 0.027 KB total ukey len = 237.615044 GB avg = 0.015 KB total ukey num = 15.751676975 Billion total entry num = 11.292990923 Billion write speed all = 32.033703288 MB/sec (with seq num) write speed all = 30.903293693 MB/sec (without seq num) 2024/11/24-07:07:13.667143 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295821: 931925 keys, 33921976 bytes, temperature: kWarm 2024/11/24-07:07:13.667214 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403233667178, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295821, "file_size": 33921976, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4432385301, "largest_seqno": 4490438566, "table_properties": {"data_size": 28026688, "index_size": 1784448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26093900, "raw_average_key_size": 28, "raw_value_size": 108123906, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931925, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403230, "oldest_key_time": 0, "file_creation_time": 1732403230, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295821, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:13.703534 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295822.sst) /dev/shm/ToplingTemp/Topling-294374-89mo1m 2024/11/24-07:07:14.708277 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295822.sst): first pass time = 1.00's, 133.589'MB/sec 2024/11/24-07:07:15.997547 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295822.sst): old prealloc_size = 147639500, real_size = 31570384 2024/11/24-07:07:15.998072 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295822.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919919 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0296 GB avg-val = 117.90 avg-zval = 32.17 2024/11/24-07:07:15.999135 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295822.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:16.806343 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295822.sst): second pass time = 1.29's, 84.192'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 44.080'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3487.132'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.12's, 286.829'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 36680.104'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.192'MB/sec zip pipeline throughput = 84.192'MB/sec entries = 919919 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.90 avg-zval = 32.17 usrkeys = 919919 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.90 avg-zval = 32.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919919 bytes = 3219728 } seq expand size = 6439433 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0296 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5129 value = 3.6654 dict = 1.00 all = 3.7648 } Zip/UnZip{ index = 0.0951 value = 0.2728 dict = 1.00 all = 0.2656 } ---------------------------- total value len = 2253.046190 GB avg = 0.199 KB (by entry num) total key len = 307.273163 GB avg = 0.027 KB total ukey len = 237.633442 GB avg = 0.015 KB total ukey num = 15.752596894 Billion total entry num = 11.293910842 Billion write speed all = 32.034187461 MB/sec (with seq num) write speed all = 30.903730804 MB/sec (without seq num) 2024/11/24-07:07:16.849718 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295822: 919919 keys, 35650498 bytes, temperature: kWarm 2024/11/24-07:07:16.849787 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403236849750, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295822, "file_size": 35650498, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4490438205, "table_properties": {"data_size": 29590144, "index_size": 1750080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25757732, "raw_average_key_size": 28, "raw_value_size": 108460002, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919919, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403233, "oldest_key_time": 0, "file_creation_time": 1732403233, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295822, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:16.887622 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295823.sst) /dev/shm/ToplingTemp/Topling-294374-1pT6pn 2024/11/24-07:07:17.799181 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295823.sst): first pass time = 0.91's, 147.246'MB/sec 2024/11/24-07:07:19.107807 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295823.sst): old prealloc_size = 147639500, real_size = 31429440 2024/11/24-07:07:19.108353 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295823.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.02 2024/11/24-07:07:19.109108 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295823.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:19.925414 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295823.sst): second pass time = 1.31's, 82.946'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.288'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3367.648'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.12's, 285.402'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35292.912'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.946'MB/sec zip pipeline throughput = 82.946'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 32.02 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4458 value = 3.6848 dict = 1.00 all = 3.7924 } Zip/UnZip{ index = 0.0957 value = 0.2714 dict = 1.00 all = 0.2637 } ---------------------------- total value len = 2253.161102 GB avg = 0.199 KB (by entry num) total key len = 307.298903 GB avg = 0.027 KB total ukey len = 237.651828 GB avg = 0.015 KB total ukey num = 15.753516194 Billion total entry num = 11.294830142 Billion write speed all = 32.034697121 MB/sec (with seq num) write speed all = 30.904192566 MB/sec (without seq num) 2024/11/24-07:07:19.976584 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295823: 919300 keys, 35391282 bytes, temperature: kWarm 2024/11/24-07:07:19.976656 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403239976619, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295823, "file_size": 35391282, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4370886504, "largest_seqno": 4483415955, "table_properties": {"data_size": 29439280, "index_size": 1760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403236, "oldest_key_time": 0, "file_creation_time": 1732403236, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295823, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:20.013157 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295824.sst) /dev/shm/ToplingTemp/Topling-294374-sCLP0n 2024/11/24-07:07:20.875862 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295824.sst): first pass time = 0.86's, 155.585'MB/sec 2024/11/24-07:07:22.214731 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295824.sst): old prealloc_size = 147639500, real_size = 32574304 2024/11/24-07:07:22.215322 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295824.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932585 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.92 avg-zval = 32.78 2024/11/24-07:07:22.216088 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295824.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:23.078024 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295824.sst): second pass time = 1.34's, 80.790'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.501'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3139.951'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.12's, 296.608'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35803.761'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.790'MB/sec zip pipeline throughput = 80.790'MB/sec entries = 932585 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.92 avg-zval = 32.78 usrkeys = 932585 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.92 avg-zval = 32.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932585 bytes = 3264064 } seq expand size = 6528095 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5150 value = 3.5367 dict = 1.00 all = 3.6583 } Zip/UnZip{ index = 0.0951 value = 0.2828 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 2253.275736 GB avg = 0.199 KB (by entry num) total key len = 307.325016 GB avg = 0.027 KB total ukey len = 237.670480 GB avg = 0.015 KB total ukey num = 15.754448779 Billion total entry num = 11.295762727 Billion write speed all = 32.035194470 MB/sec (with seq num) write speed all = 30.904641164 MB/sec (without seq num) 2024/11/24-07:07:23.126266 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295824: 932585 keys, 36688574 bytes, temperature: kWarm 2024/11/24-07:07:23.126342 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403243126305, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295824, "file_size": 36688574, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4490439419, "table_properties": {"data_size": 30567120, "index_size": 1773824, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26112380, "raw_average_key_size": 28, "raw_value_size": 108105426, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932585, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403240, "oldest_key_time": 0, "file_creation_time": 1732403240, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295824, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:23.163034 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295825.sst) /dev/shm/ToplingTemp/Topling-294374-mEhN7m 2024/11/24-07:07:24.166823 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295825.sst): first pass time = 1.00's, 133.718'MB/sec 2024/11/24-07:07:25.465773 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295825.sst): old prealloc_size = 147639500, real_size = 31441568 2024/11/24-07:07:25.466319 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295825.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 32.02 2024/11/24-07:07:25.467070 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295825.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:26.279762 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295825.sst): second pass time = 1.30's, 83.564'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.495'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3385.742'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.12's, 284.509'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 26396.701'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.564'MB/sec zip pipeline throughput = 83.564'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 32.02 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0354 GB } UnZip/Zip{ index = 10.3768 value = 3.6847 dict = 1.00 all = 3.7911 } Zip/UnZip{ index = 0.0964 value = 0.2714 dict = 1.00 all = 0.2638 } ---------------------------- total value len = 2253.390648 GB avg = 0.199 KB (by entry num) total key len = 307.350756 GB avg = 0.027 KB total ukey len = 237.688866 GB avg = 0.015 KB total ukey num = 15.755368079 Billion total entry num = 11.296682027 Billion write speed all = 32.035670913 MB/sec (with seq num) write speed all = 30.905070885 MB/sec (without seq num) 2024/11/24-07:07:26.325919 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295825: 919300 keys, 35403378 bytes, temperature: kWarm 2024/11/24-07:07:26.326008 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403246325953, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295825, "file_size": 35403378, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4310620866, "largest_seqno": 4421379857, "table_properties": {"data_size": 29439696, "index_size": 1771840, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403243, "oldest_key_time": 0, "file_creation_time": 1732403243, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295825, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:26.366031 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295826.sst) /dev/shm/ToplingTemp/Topling-294374-hU1Hpn 2024/11/24-07:07:27.216012 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295826.sst): first pass time = 0.85's, 157.913'MB/sec 2024/11/24-07:07:28.516627 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295826.sst): old prealloc_size = 147639500, real_size = 30444960 2024/11/24-07:07:28.517175 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295826.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929375 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.42 avg-zval = 30.59 2024/11/24-07:07:28.517918 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295826.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:29.332481 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295826.sst): second pass time = 1.30's, 83.271'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.206'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3380.193'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.336'MB/sec write SST data time = 0.12's, 277.263'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 30783.903'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.271'MB/sec zip pipeline throughput = 83.271'MB/sec entries = 929375 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.42 avg-zval = 30.59 usrkeys = 929375 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.42 avg-zval = 30.59 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929375 bytes = 3136656 } seq expand size = 6505625 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.4441 value = 3.8053 dict = 1.00 all = 3.8980 } Zip/UnZip{ index = 0.0957 value = 0.2628 dict = 1.00 all = 0.2565 } ---------------------------- total value len = 2253.505349 GB avg = 0.199 KB (by entry num) total key len = 307.376779 GB avg = 0.027 KB total ukey len = 237.707453 GB avg = 0.015 KB total ukey num = 15.756297454 Billion total entry num = 11.297611402 Billion write speed all = 32.036207933 MB/sec (with seq num) write speed all = 30.905558071 MB/sec (without seq num) 2024/11/24-07:07:29.380599 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295826: 929375 keys, 34432608 bytes, temperature: kWarm 2024/11/24-07:07:29.380669 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403249380634, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295826, "file_size": 34432608, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4421379858, "largest_seqno": 4490438595, "table_properties": {"data_size": 28432720, "index_size": 1779712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26022500, "raw_average_key_size": 28, "raw_value_size": 108195326, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929375, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403246, "oldest_key_time": 0, "file_creation_time": 1732403246, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295826, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:29.419030 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295827.sst) /dev/shm/ToplingTemp/Topling-294374-5HS7Do 2024/11/24-07:07:30.346751 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295827.sst): first pass time = 0.93's, 144.681'MB/sec 2024/11/24-07:07:31.658283 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295827.sst): old prealloc_size = 147639500, real_size = 32414752 2024/11/24-07:07:31.658862 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295827.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922487 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0304 GB avg-val = 117.50 avg-zval = 32.97 2024/11/24-07:07:31.659635 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295827.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:32.483737 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295827.sst): second pass time = 1.31's, 82.722'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 44.222'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3175.871'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 291.777'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 30750.440'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.722'MB/sec zip pipeline throughput = 82.722'MB/sec entries = 922487 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.50 avg-zval = 32.97 usrkeys = 922487 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.50 avg-zval = 32.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922487 bytes = 3228720 } seq expand size = 6457409 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.4444 value = 3.5634 dict = 1.00 all = 3.6770 } Zip/UnZip{ index = 0.0957 value = 0.2806 dict = 1.00 all = 0.2720 } ---------------------------- total value len = 2253.620195 GB avg = 0.199 KB (by entry num) total key len = 307.402608 GB avg = 0.027 KB total ukey len = 237.725903 GB avg = 0.015 KB total ukey num = 15.757219941 Billion total entry num = 11.298533889 Billion write speed all = 32.036704814 MB/sec (with seq num) write speed all = 30.906007205 MB/sec (without seq num) 2024/11/24-07:07:32.530405 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295827: 922487 keys, 36501668 bytes, temperature: kWarm 2024/11/24-07:07:32.530480 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403252530444, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295827, "file_size": 36501668, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4490439278, "table_properties": {"data_size": 30417488, "index_size": 1766464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25829636, "raw_average_key_size": 28, "raw_value_size": 108388202, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922487, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403249, "oldest_key_time": 0, "file_creation_time": 1732403249, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295827, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:32.566952 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295828.sst) /dev/shm/ToplingTemp/Topling-294374-HvY04l 2024/11/24-07:07:33.574591 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295828.sst): first pass time = 1.01's, 133.207'MB/sec 2024/11/24-07:07:34.873081 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295828.sst): old prealloc_size = 147639500, real_size = 31321872 2024/11/24-07:07:34.873614 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295828.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.91 2024/11/24-07:07:34.874344 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295828.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:35.686063 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295828.sst): second pass time = 1.30's, 83.617'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.402'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3436.611'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.12's, 285.676'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31812.877'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.617'MB/sec zip pipeline throughput = 83.617'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.91 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4462 value = 3.6983 dict = 1.00 all = 3.8040 } Zip/UnZip{ index = 0.0957 value = 0.2704 dict = 1.00 all = 0.2629 } ---------------------------- total value len = 2253.735107 GB avg = 0.199 KB (by entry num) total key len = 307.428349 GB avg = 0.027 KB total ukey len = 237.744289 GB avg = 0.015 KB total ukey num = 15.758139241 Billion total entry num = 11.299453189 Billion write speed all = 32.037180911 MB/sec (with seq num) write speed all = 30.906436600 MB/sec (without seq num) 2024/11/24-07:07:35.730004 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295828: 919300 keys, 35283698 bytes, temperature: kWarm 2024/11/24-07:07:35.730076 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403255730039, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295828, "file_size": 35283698, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4359887807, "largest_seqno": 4472145366, "table_properties": {"data_size": 29331776, "index_size": 1760064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403252, "oldest_key_time": 0, "file_creation_time": 1732403252, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295828, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:35.765992 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295829.sst) /dev/shm/ToplingTemp/Topling-294374-Tk01Kp 2024/11/24-07:07:36.725575 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295829.sst): first pass time = 0.96's, 139.877'MB/sec 2024/11/24-07:07:38.044523 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295829.sst): old prealloc_size = 147639500, real_size = 32359024 2024/11/24-07:07:38.045092 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295829.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932700 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.90 avg-zval = 32.54 2024/11/24-07:07:38.045879 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295829.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:38.892315 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295829.sst): second pass time = 1.32's, 82.008'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.024'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3240.327'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.12's, 295.536'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31133.928'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.008'MB/sec zip pipeline throughput = 82.008'MB/sec entries = 932700 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.90 avg-zval = 32.54 usrkeys = 932700 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.90 avg-zval = 32.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932700 bytes = 3264464 } seq expand size = 6528900 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5144 value = 3.5617 dict = 1.00 all = 3.6798 } Zip/UnZip{ index = 0.0951 value = 0.2808 dict = 1.00 all = 0.2718 } ---------------------------- total value len = 2253.849739 GB avg = 0.199 KB (by entry num) total key len = 307.454464 GB avg = 0.027 KB total ukey len = 237.762943 GB avg = 0.015 KB total ukey num = 15.759071941 Billion total entry num = 11.300385889 Billion write speed all = 32.037656572 MB/sec (with seq num) write speed all = 30.906864277 MB/sec (without seq num) 2024/11/24-07:07:38.938526 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295829: 932700 keys, 36474126 bytes, temperature: kWarm 2024/11/24-07:07:38.938598 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403258938561, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295829, "file_size": 36474126, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679961, "largest_seqno": 4490439345, "table_properties": {"data_size": 30351520, "index_size": 1774144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26115600, "raw_average_key_size": 28, "raw_value_size": 108102256, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932700, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403255, "oldest_key_time": 0, "file_creation_time": 1732403255, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295829, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:38.976670 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295830.sst) /dev/shm/ToplingTemp/Topling-294374-XcyIRp 2024/11/24-07:07:39.970392 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295830.sst): first pass time = 0.99's, 135.071'MB/sec 2024/11/24-07:07:41.265474 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295830.sst): old prealloc_size = 147639500, real_size = 31256896 2024/11/24-07:07:41.266020 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295830.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.84 2024/11/24-07:07:41.267946 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295830.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:42.074546 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295830.sst): second pass time = 1.29's, 83.849'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.531'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3353.247'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 277.462'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35187.611'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.849'MB/sec zip pipeline throughput = 83.849'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.84 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4439 value = 3.7065 dict = 1.00 all = 3.8110 } Zip/UnZip{ index = 0.0957 value = 0.2698 dict = 1.00 all = 0.2624 } ---------------------------- total value len = 2253.964651 GB avg = 0.199 KB (by entry num) total key len = 307.480205 GB avg = 0.027 KB total ukey len = 237.781329 GB avg = 0.015 KB total ukey num = 15.759991241 Billion total entry num = 11.301305189 Billion write speed all = 32.038140643 MB/sec (with seq num) write speed all = 30.907301369 MB/sec (without seq num) 2024/11/24-07:07:42.119171 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295830: 919300 keys, 35218738 bytes, temperature: kWarm 2024/11/24-07:07:42.119244 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403262119206, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295830, "file_size": 35218738, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4299337021, "largest_seqno": 4409685763, "table_properties": {"data_size": 29266416, "index_size": 1760448, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403258, "oldest_key_time": 0, "file_creation_time": 1732403258, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295830, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:42.156653 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295831.sst) /dev/shm/ToplingTemp/Topling-294374-4pCPnm 2024/11/24-07:07:43.164514 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295831.sst): first pass time = 1.01's, 133.177'MB/sec 2024/11/24-07:07:44.470677 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295831.sst): old prealloc_size = 147639500, real_size = 30650944 2024/11/24-07:07:44.471240 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295831.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926680 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0286 GB avg-val = 116.84 avg-zval = 30.91 2024/11/24-07:07:44.472024 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295831.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:45.282086 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295831.sst): second pass time = 1.31's, 82.943'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 42.682'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3250.240'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.339'MB/sec write SST data time = 0.13's, 272.998'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 31614.124'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.943'MB/sec zip pipeline throughput = 82.943'MB/sec entries = 926680 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.84 avg-zval = 30.91 usrkeys = 926680 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.84 avg-zval = 30.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926680 bytes = 3127552 } seq expand size = 6486760 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0286 GB dict = 1.08 MB all = 0.0346 GB } UnZip/Zip{ index = 10.4450 value = 3.7798 dict = 1.00 all = 3.8756 } Zip/UnZip{ index = 0.0957 value = 0.2646 dict = 1.00 all = 0.2580 } ---------------------------- total value len = 2254.079409 GB avg = 0.199 KB (by entry num) total key len = 307.506152 GB avg = 0.027 KB total ukey len = 237.799862 GB avg = 0.015 KB total ukey num = 15.760917921 Billion total entry num = 11.302231869 Billion write speed all = 32.038615186 MB/sec (with seq num) write speed all = 30.907728556 MB/sec (without seq num) 2024/11/24-07:07:45.325042 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295831: 926680 keys, 34631498 bytes, temperature: kWarm 2024/11/24-07:07:45.325114 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403265325078, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295831, "file_size": 34631498, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4409685764, "largest_seqno": 4490438624, "table_properties": {"data_size": 28644720, "index_size": 1774400, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947040, "raw_average_key_size": 28, "raw_value_size": 108270700, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926680, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403262, "oldest_key_time": 0, "file_creation_time": 1732403262, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295831, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:45.364682 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295832.sst) /dev/shm/ToplingTemp/Topling-294374-MfGIlo 2024/11/24-07:07:46.380508 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295832.sst): first pass time = 1.02's, 132.133'MB/sec 2024/11/24-07:07:47.692179 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295832.sst): old prealloc_size = 147639500, real_size = 32862720 2024/11/24-07:07:47.692727 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295832.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925387 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.04 avg-zval = 33.35 2024/11/24-07:07:47.693483 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295832.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:48.537375 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295832.sst): second pass time = 1.31's, 82.622'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.727'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3385.051'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 291.592'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32391.641'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.622'MB/sec zip pipeline throughput = 82.622'MB/sec entries = 925387 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.04 avg-zval = 33.35 usrkeys = 925387 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.04 avg-zval = 33.35 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925387 bytes = 3238864 } seq expand size = 6477709 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4462 value = 3.5097 dict = 1.00 all = 3.6318 } Zip/UnZip{ index = 0.0957 value = 0.2849 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 2254.194193 GB avg = 0.199 KB (by entry num) total key len = 307.532063 GB avg = 0.027 KB total ukey len = 237.818370 GB avg = 0.015 KB total ukey num = 15.761843308 Billion total entry num = 11.303157256 Billion write speed all = 32.039070441 MB/sec (with seq num) write speed all = 30.908137264 MB/sec (without seq num) 2024/11/24-07:07:48.583893 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295832: 925387 keys, 36956374 bytes, temperature: kWarm 2024/11/24-07:07:48.583962 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403268583928, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295832, "file_size": 36956374, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679976, "largest_seqno": 4490438151, "table_properties": {"data_size": 30859504, "index_size": 1771712, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25910836, "raw_average_key_size": 28, "raw_value_size": 108306966, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925387, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403265, "oldest_key_time": 0, "file_creation_time": 1732403265, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295832, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:48.632368 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295833.sst) /dev/shm/ToplingTemp/Topling-294374-POAmVo 2024/11/24-07:07:49.602608 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295833.sst): first pass time = 0.97's, 138.341'MB/sec 2024/11/24-07:07:50.894663 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295833.sst): old prealloc_size = 147639500, real_size = 31344768 2024/11/24-07:07:50.895214 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295833.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.93 2024/11/24-07:07:50.896134 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295833.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-07:07:51.709646 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295833.sst): second pass time = 1.29's, 84.041'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.323'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3339.971'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 277.864'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31479.676'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.041'MB/sec zip pipeline throughput = 84.041'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.92 avg-val = 118.00 avg-zval = 31.93 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.92 avg-val = 125.00 avg-zval = 31.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4436 value = 3.6955 dict = 1.00 all = 3.8015 } Zip/UnZip{ index = 0.0958 value = 0.2706 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 2254.309106 GB avg = 0.199 KB (by entry num) total key len = 307.557803 GB avg = 0.027 KB total ukey len = 237.836756 GB avg = 0.015 KB total ukey num = 15.762762608 Billion total entry num = 11.304076556 Billion write speed all = 32.039558391 MB/sec (with seq num) write speed all = 30.908578106 MB/sec (without seq num) 2024/11/24-07:07:51.755041 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295833: 919300 keys, 35306610 bytes, temperature: kWarm 2024/11/24-07:07:51.755118 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403271755078, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295833, "file_size": 35306610, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4347348637, "largest_seqno": 4459500007, "table_properties": {"data_size": 29354224, "index_size": 1760512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403268, "oldest_key_time": 0, "file_creation_time": 1732403268, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295833, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:51.794836 139957626406464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295834.sst) /dev/shm/ToplingTemp/Topling-294374-WwG4Sn 2024/11/24-07:07:52.508032 139957626406464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295834.sst): first pass time = 0.71's, 142.996'MB/sec 2024/11/24-07:07:53.508039 139957626406464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295834.sst): old prealloc_size = 147639500, real_size = 23469744 2024/11/24-07:07:53.508512 139957626406464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295834.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 711187 prefix = 4 raw-key = 0.0114 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0821 GB zip-val = 0.0219 GB avg-val = 115.39 avg-zval = 30.82 2024/11/24-07:07:53.509110 139957626406464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295834.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0018 GB, waited 0.000 sec, Key+Value = 0.102 GB 2024/11/24-07:07:54.158523 139957626406464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00/295834.sst): second pass time = 1.00's, 82.153'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.65's, 40.634'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3066.467'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.321'MB/sec write SST data time = 0.10's, 276.800'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.824'MB, throughput = 24780.755'MB/sec zip my value time = 1.00's, unzip length = 0.082'GB zip my value throughput = 82.153'MB/sec zip pipeline throughput = 82.153'MB/sec entries = 711187 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.39 avg-zval = 30.82 usrkeys = 711187 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.39 avg-zval = 30.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 711187 bytes = 2311376 } seq expand size = 4978309 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0142 GB value = 0.0821 GB dict = 0.82 MB all = 0.1020 GB } __ZipSize{ index = 0.0014 GB value = 0.0219 GB dict = 0.82 MB all = 0.0264 GB } UnZip/Zip{ index = 10.3751 value = 3.7437 dict = 1.00 all = 3.8583 } Zip/UnZip{ index = 0.0964 value = 0.2671 dict = 1.00 all = 0.2592 } ---------------------------- total value len = 2254.396148 GB avg = 0.199 KB (by entry num) total key len = 307.577716 GB avg = 0.027 KB total ukey len = 237.850980 GB avg = 0.015 KB total ukey num = 15.763473795 Billion total entry num = 11.304787743 Billion write speed all = 32.039914742 MB/sec (with seq num) write speed all = 30.908897941 MB/sec (without seq num) 2024/11/24-07:07:54.194633 139957626406464 [compaction_job.cc:1948] [default] [JOB 43189] Generated table #295834: 711187 keys, 26430884 bytes, temperature: kWarm 2024/11/24-07:07:54.194705 139957626406464 EVENT_LOG_v1 {"time_micros": 1732403274194669, "cf_name": "default", "job": 43189, "event": "table_file_creation", "file_number": 295834, "file_size": 26430884, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4437459339, "largest_seqno": 4490025044, "table_properties": {"data_size": 21920800, "index_size": 1370944, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 19913236, "raw_average_key_size": 28, "raw_value_size": 82064006, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 711187, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732403271, "oldest_key_time": 0, "file_creation_time": 1732403271, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 295834, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-07:07:54.216356 139957626406464 [compaction_job.cc:703] job-43189: subcompact[0], size: 0.944101 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43189/att-00 [295808,295809,295810,295811,295812,295813,295814,295815,295816,295817,295818,295819,295820,295821,295822,295823,295824,295825,295826,295827,295828,295829,295830,295831,295832,295833,295834] 2024/11/24-07:07:54.417681 139957626406464 (Original Log Time 2024/11/24-07:06:23.288861) EVENT_LOG_v1 {"time_micros": 1732403183288820, "job": 43189, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [295802], "files_L3": [295743, 295752, 295753, 295754, 295755, 295756, 295757, 295758, 295759, 295760, 295761, 295762, 295763, 295764, 295765, 295766, 295767, 295768, 295769, 295770, 295771, 295772, 295773, 295774, 295775, 295776, 295777], "score": 0.930917, "input_data_size": 966589771, "oldest_snapshot_seqno": 4490449214} 2024/11/24-07:07:54.417685 139957626406464 (Original Log Time 2024/11/24-07:07:54.416001) [compaction_job.cc:2022] [default] [JOB 43189] Compacted 1@2 + 27@3 files to L3 => 944101348 bytes 2024/11/24-07:07:54.417687 139957626406464 (Original Log Time 2024/11/24-07:07:54.417606) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 27 0 0 0] max score 0.04, MB/sec: 10.6 rd, 10.4 wr, level 3, files in(1, 27) out(27 +0 blob) MB in(27.1, 894.7 +0.0 blob) out(900.4 +0.0 blob), read-write-amplify(67.3) write-amplify(33.2) OK, records in: 24913135, records dropped: 100503 output_compression: Snappy 2024/11/24-07:07:54.417690 139957626406464 (Original Log Time 2024/11/24-07:07:54.417654) EVENT_LOG_v1 {"time_micros": 1732403274417630, "cf": "default", "job": 43189, "event": "compaction_finished", "compaction_time_micros": 90927506, "compaction_time_cpu_micros": 38193652, "output_level": 3, "num_output_files": 27, "total_output_size": 944101348, "num_input_records": 24913135, "num_output_records": 24812632, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 663849071, "file_range_sync_nanos": 0, "file_fsync_nanos": 20333, "file_prepare_write_nanos": 64382, "lsm_state": [0, 0, 0, 27, 0, 0, 0]} 2024/11/24-07:07:54.570220 139957626406464 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732403183,"executesMs":91253,"compactionJobId":43189,"attempt":0,"compactionInputRawBytes":3605911558,"compactionInputZipBytes":966589771,"compactionOutputRawBytes":0,"compactionOutputZipBytes":944101348,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-b18c8aad17a7508fbe06cab4c5e276b2-a824ba85a33fb1e4-00"}}, response = 2024/11/24-07:07:54.570294 139957626406464 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-43189/att-00: olev 3, work 91.254 s, result 7.755 ms, install 1.705 ms, input{raw 3.358 GiB zip 921.812 MiB}