2024/11/23-17:59:52.201423 139957265520192 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00: fork to child time = 0.007635 sec 2024/11/23-17:59:52.202164 139957265520192 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00 2024/11/23-17:59:52.209334 139957265520192 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00 2024/11/23-17:59:52.210235 139957265520192 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-17:59:52.210508 139957265520192 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-17:59:52.287328 139957265520192 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32564/att-00/MANIFEST-206347 2024/11/23-17:59:52.320665 139957265520192 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32564/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 273656, last_sequence is 4166734235, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 273496 2024/11/23-17:59:52.320682 139957265520192 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-17:59:52.320860 139957265520192 [version_set.cc:5721] Creating manifest 273656 2024/11/23-17:59:52.356831 139957265520192 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-17:59:52.381115 139957265520192 [compaction_job.cc:2437] [default] [JOB 32564] Compacting 1@2 + 21@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-17:59:52.381144 139957265520192 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 24378 Base level 2, inputs: [273621(27MB)], [273643(37MB) 273644(3265KB) 273569(39MB) 273570(38MB) 273571(35MB) 273572(33MB) 273573(34MB) 273574(38MB) 273575(35MB) 273576(35MB) 273577(33MB) 273578(35MB) 273579(36MB) 273580(36MB) 273581(35MB) 273582(35MB) 273583(36MB) 273584(37MB) 273589(37MB) 273590(37MB) 273649(36MB)] 2024/11/23-17:59:52.390090 139957265520192 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:59:52.390133 139957265520192 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:59:52.390162 139957265520192 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-17:59:52.390172 139957265520192 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:59:52.390190 139957265520192 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-17:59:53.175735 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273657.sst) /dev/shm/ToplingTemp/Topling-141995-NI850b 2024/11/23-17:59:53.735560 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273657.sst): first pass time = 0.56's, 239.770'MB/sec 2024/11/23-17:59:55.260264 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273657.sst): old prealloc_size = 147639500, real_size = 35452064 2024/11/23-17:59:55.260872 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273657.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925619 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0334 GB avg-val = 117.00 avg-zval = 36.14 2024/11/23-17:59:55.261818 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273657.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-17:59:56.311802 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273657.sst): second pass time = 1.52's, 71.062'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 37.499'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3100.530'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.267'MB/sec write SST data time = 0.18's, 215.648'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 36305.433'MB/sec zip my value time = 1.52's, unzip length = 0.108'GB zip my value throughput = 71.062'MB/sec zip pipeline throughput = 79.907'MB/sec entries = 925619 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.00 avg-zval = 36.14 usrkeys = 925619 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.00 avg-zval = 36.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925619 bytes = 3123984 } seq expand size = 6479333 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0394 GB } UnZip/Zip{ index = 10.4455 value = 3.2379 dict = 1.00 all = 3.4039 } Zip/UnZip{ index = 0.0957 value = 0.3088 dict = 1.00 all = 0.2938 } ---------------------------- total value len = 8.963400 GB avg = 0.168 KB (by entry num) total key len = 1.466738 GB avg = 0.028 KB total ukey len = 1.016390 GB avg = 0.015 KB total ukey num = 0.066476054 Billion total entry num = 0.053283783 Billion write speed all = 0.319119663 MB/sec (with seq num) write speed all = 0.306077532 MB/sec (without seq num) 2024/11/23-17:59:56.361972 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273657: 925619 keys, 39430966 bytes, temperature: kWarm 2024/11/23-17:59:56.362086 139957265520192 EVENT_LOG_v1 {"time_micros": 1732355996362028, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273657, "file_size": 39430966, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4051668904, "largest_seqno": 4166409903, "table_properties": {"data_size": 33448208, "index_size": 1772288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25917332, "raw_average_key_size": 28, "raw_value_size": 108300534, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925619, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355993, "oldest_key_time": 0, "file_creation_time": 1732355993, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273657, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-17:59:56.399454 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273658.sst) /dev/shm/ToplingTemp/Topling-141995-iLeEZc 2024/11/23-17:59:57.401786 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273658.sst): first pass time = 1.00's, 133.911'MB/sec 2024/11/23-17:59:59.124859 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273658.sst): old prealloc_size = 147639500, real_size = 35916720 2024/11/23-17:59:59.125471 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273658.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932540 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0339 GB avg-val = 115.93 avg-zval = 36.36 2024/11/23-17:59:59.126333 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273658.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:00.243903 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273658.sst): second pass time = 1.72's, 62.780'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.12's, 35.881'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3010.468'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.256'MB/sec write SST data time = 0.21's, 194.378'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 26050.986'MB/sec zip my value time = 1.72's, unzip length = 0.108'GB zip my value throughput = 62.780'MB/sec zip pipeline throughput = 62.780'MB/sec entries = 932540 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.93 avg-zval = 36.36 usrkeys = 932540 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.93 avg-zval = 36.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932540 bytes = 3380480 } seq expand size = 6527780 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0339 GB dict = 1.08 MB all = 0.0402 GB } UnZip/Zip{ index = 10.5141 value = 3.1881 dict = 1.00 all = 3.3428 } Zip/UnZip{ index = 0.0951 value = 0.3137 dict = 1.00 all = 0.2992 } ---------------------------- total value len = 9.078034 GB avg = 0.167 KB (by entry num) total key len = 1.492850 GB avg = 0.028 KB total ukey len = 1.035041 GB avg = 0.015 KB total ukey num = 0.067408594 Billion total entry num = 0.054216323 Billion write speed all = 0.323386998 MB/sec (with seq num) write speed all = 0.310118209 MB/sec (without seq num) 2024/11/23-18:00:00.295630 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273658: 932540 keys, 40151838 bytes, temperature: kWarm 2024/11/23-18:00:00.295702 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356000295665, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273658, "file_size": 40151838, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3736072251, "largest_seqno": 4166409544, "table_properties": {"data_size": 33909536, "index_size": 1773888, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26111120, "raw_average_key_size": 28, "raw_value_size": 108106688, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932540, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732355996, "oldest_key_time": 0, "file_creation_time": 1732355996, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273658, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:00.335908 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273659.sst) /dev/shm/ToplingTemp/Topling-141995-nIH8Cc 2024/11/23-18:00:01.669564 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273659.sst): first pass time = 1.33's, 100.642'MB/sec 2024/11/23-18:00:03.285794 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273659.sst): old prealloc_size = 147639500, real_size = 36660816 2024/11/23-18:00:03.286415 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273659.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932449 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1081 GB zip-val = 0.0346 GB avg-val = 115.94 avg-zval = 37.14 2024/11/23-18:00:03.287172 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273659.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:04.067401 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273659.sst): second pass time = 1.61's, 66.945'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.78's, 51.187'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2949.050'MB/sec (index lex order gen) rebuild zvType time = 0.65's, 0.358'MB/sec write SST data time = 0.13's, 309.101'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 37896.635'MB/sec zip my value time = 1.61's, unzip length = 0.108'GB zip my value throughput = 66.945'MB/sec zip pipeline throughput = 66.945'MB/sec entries = 932449 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.94 avg-zval = 37.14 usrkeys = 932449 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.64 avg-zval = 37.14 TagRS{ kinds = RS_Key0_TagN bytes = 145800 } TagArray{ size = 625657 bytes = 2346224 } seq expand size = 4379599 multi value expand size = 0 cnt WriteAppend = 19 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.3779 value = 3.1218 dict = 1.00 all = 3.3548 } Zip/UnZip{ index = 0.0964 value = 0.3203 dict = 1.00 all = 0.2981 } ---------------------------- total value len = 9.190523 GB avg = 0.167 KB (by entry num) total key len = 1.518958 GB avg = 0.028 KB total ukey len = 1.053690 GB avg = 0.015 KB total ukey num = 0.068341043 Billion total entry num = 0.055148772 Billion write speed all = 0.327588687 MB/sec (with seq num) write speed all = 0.314093270 MB/sec (without seq num) 2024/11/23-18:00:04.118246 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273659: 932449 keys, 40007254 bytes, temperature: kWarm 2024/11/23-18:00:04.118319 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356004118286, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273659, "file_size": 40007254, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776165, "table_properties": {"data_size": 34630528, "index_size": 1796992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26108572, "raw_average_key_size": 28, "raw_value_size": 108109286, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932449, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356000, "oldest_key_time": 0, "file_creation_time": 1732356000, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273659, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:04.156486 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273660.sst) /dev/shm/ToplingTemp/Topling-141995-P7XAEc 2024/11/23-18:00:05.357782 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273660.sst): first pass time = 1.20's, 111.731'MB/sec 2024/11/23-18:00:06.701259 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273660.sst): old prealloc_size = 147639500, real_size = 33216528 2024/11/23-18:00:06.701849 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273660.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926834 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.81 avg-zval = 33.67 2024/11/23-18:00:06.702677 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273660.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:07.571136 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273660.sst): second pass time = 1.34's, 80.628'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 43.164'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.258'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.315'MB/sec write SST data time = 0.13's, 280.057'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 27832.810'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.628'MB/sec zip pipeline throughput = 80.628'MB/sec entries = 926834 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.81 avg-zval = 33.67 usrkeys = 926834 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.81 avg-zval = 33.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926834 bytes = 3475648 } seq expand size = 6487838 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.4445 value = 3.4690 dict = 1.00 all = 3.5747 } Zip/UnZip{ index = 0.0957 value = 0.2883 dict = 1.00 all = 0.2797 } ---------------------------- total value len = 9.305277 GB avg = 0.166 KB (by entry num) total key len = 1.544909 GB avg = 0.028 KB total ukey len = 1.072227 GB avg = 0.015 KB total ukey num = 0.069267877 Billion total entry num = 0.056075606 Billion write speed all = 0.331857117 MB/sec (with seq num) write speed all = 0.318136366 MB/sec (without seq num) 2024/11/23-18:00:07.620372 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273660: 926834 keys, 37546542 bytes, temperature: kWarm 2024/11/23-18:00:07.620442 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356007620406, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273660, "file_size": 37546542, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3277180002, "largest_seqno": 4012775902, "table_properties": {"data_size": 31209856, "index_size": 1774784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25951352, "raw_average_key_size": 28, "raw_value_size": 108266424, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926834, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356004, "oldest_key_time": 0, "file_creation_time": 1732356004, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273660, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:07.657872 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273661.sst) /dev/shm/ToplingTemp/Topling-141995-DTo6hd 2024/11/23-18:00:08.544422 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273661.sst): first pass time = 0.89's, 151.400'MB/sec 2024/11/23-18:00:10.200544 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273661.sst): old prealloc_size = 147639500, real_size = 31751280 2024/11/23-18:00:10.201130 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273661.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932240 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0297 GB avg-val = 115.97 avg-zval = 31.91 2024/11/23-18:00:10.201930 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273661.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:11.051028 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273661.sst): second pass time = 1.66's, 65.314'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.173'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3126.508'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 268.950'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 36380.975'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.314'MB/sec zip pipeline throughput = 98.473'MB/sec entries = 932240 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.97 avg-zval = 31.91 usrkeys = 932240 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.97 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932240 bytes = 3262848 } seq expand size = 6525680 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5126 value = 3.6348 dict = 1.00 all = 3.7423 } Zip/UnZip{ index = 0.0951 value = 0.2751 dict = 1.00 all = 0.2672 } ---------------------------- total value len = 9.419918 GB avg = 0.165 KB (by entry num) total key len = 1.571012 GB avg = 0.028 KB total ukey len = 1.090872 GB avg = 0.016 KB total ukey num = 0.070200117 Billion total entry num = 0.057007846 Billion write speed all = 0.336126034 MB/sec (with seq num) write speed all = 0.322178664 MB/sec (without seq num) 2024/11/23-18:00:11.103000 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273661: 932240 keys, 35864710 bytes, temperature: kWarm 2024/11/23-18:00:11.103093 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356011103043, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273661, "file_size": 35864710, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3807172320, "largest_seqno": 4012775918, "table_properties": {"data_size": 29744480, "index_size": 1773568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102720, "raw_average_key_size": 28, "raw_value_size": 108115060, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932240, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356007, "oldest_key_time": 0, "file_creation_time": 1732356007, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273661, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:11.185895 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273662.sst) /dev/shm/ToplingTemp/Topling-141995-CVYufa 2024/11/23-18:00:12.244756 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273662.sst): first pass time = 1.06's, 126.763'MB/sec 2024/11/23-18:00:13.596134 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273662.sst): old prealloc_size = 147639500, real_size = 32573152 2024/11/23-18:00:13.596695 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273662.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925338 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 117.05 avg-zval = 33.05 2024/11/23-18:00:13.597535 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273662.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:14.433172 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273662.sst): second pass time = 1.35's, 80.195'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.083'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3287.304'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 285.862'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33515.153'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.195'MB/sec zip pipeline throughput = 80.195'MB/sec entries = 925338 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.05 avg-zval = 33.05 usrkeys = 925338 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.05 avg-zval = 33.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925338 bytes = 3470032 } seq expand size = 6477366 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5137 value = 3.5416 dict = 1.00 all = 3.6375 } Zip/UnZip{ index = 0.0951 value = 0.2824 dict = 1.00 all = 0.2749 } ---------------------------- total value len = 9.534704 GB avg = 0.165 KB (by entry num) total key len = 1.596922 GB avg = 0.028 KB total ukey len = 1.109378 GB avg = 0.016 KB total ukey num = 0.071125455 Billion total entry num = 0.057933184 Billion write speed all = 0.340393583 MB/sec (with seq num) write speed all = 0.326221289 MB/sec (without seq num) 2024/11/23-18:00:14.478489 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273662: 925338 keys, 36898108 bytes, temperature: kWarm 2024/11/23-18:00:14.478561 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356014478524, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273662, "file_size": 36898108, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 4012775935, "table_properties": {"data_size": 30581392, "index_size": 1760256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25909464, "raw_average_key_size": 28, "raw_value_size": 108308324, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925338, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356011, "oldest_key_time": 0, "file_creation_time": 1732356011, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273662, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:14.516541 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273663.sst) /dev/shm/ToplingTemp/Topling-141995-xv86Hb 2024/11/23-18:00:15.587649 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273663.sst): first pass time = 1.07's, 125.311'MB/sec 2024/11/23-18:00:16.955341 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273663.sst): old prealloc_size = 147639500, real_size = 35785616 2024/11/23-18:00:16.955959 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273663.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927272 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0338 GB avg-val = 116.74 avg-zval = 36.43 2024/11/23-18:00:16.956791 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273663.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:17.794030 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273663.sst): second pass time = 1.37's, 79.232'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.697'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2957.424'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 306.459'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35481.391'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.232'MB/sec zip pipeline throughput = 79.232'MB/sec entries = 927272 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.74 avg-zval = 36.43 usrkeys = 927272 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.74 avg-zval = 36.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927272 bytes = 3361376 } seq expand size = 6490904 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0338 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4438 value = 3.2049 dict = 1.00 all = 3.3553 } Zip/UnZip{ index = 0.0958 value = 0.3120 dict = 1.00 all = 0.2980 } ---------------------------- total value len = 9.649449 GB avg = 0.164 KB (by entry num) total key len = 1.622885 GB avg = 0.028 KB total ukey len = 1.127924 GB avg = 0.016 KB total ukey num = 0.072052727 Billion total entry num = 0.058860456 Billion write speed all = 0.344660888 MB/sec (with seq num) write speed all = 0.330263233 MB/sec (without seq num) 2024/11/23-18:00:17.844876 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273663: 927272 keys, 40001136 bytes, temperature: kWarm 2024/11/23-18:00:17.844949 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356017844911, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273663, "file_size": 40001136, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3514720078, "largest_seqno": 4012775967, "table_properties": {"data_size": 33777856, "index_size": 1775744, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25963616, "raw_average_key_size": 28, "raw_value_size": 108254172, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927272, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356014, "oldest_key_time": 0, "file_creation_time": 1732356014, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273663, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:17.880856 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273664.sst) /dev/shm/ToplingTemp/Topling-141995-TzssGd 2024/11/23-18:00:18.845084 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273664.sst): first pass time = 0.96's, 139.202'MB/sec 2024/11/23-18:00:20.178786 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273664.sst): old prealloc_size = 147639500, real_size = 32479232 2024/11/23-18:00:20.179377 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273664.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928520 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0305 GB avg-val = 116.55 avg-zval = 32.83 2024/11/23-18:00:20.180212 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273664.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:21.015261 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273664.sst): second pass time = 1.33's, 81.208'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.013'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3107.696'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.328'MB/sec write SST data time = 0.13's, 286.226'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33584.323'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.208'MB/sec zip pipeline throughput = 81.208'MB/sec entries = 928520 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.55 avg-zval = 32.83 usrkeys = 928520 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.55 avg-zval = 32.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928520 bytes = 3481968 } seq expand size = 6499640 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5147 value = 3.5504 dict = 1.00 all = 3.6458 } Zip/UnZip{ index = 0.0951 value = 0.2817 dict = 1.00 all = 0.2743 } ---------------------------- total value len = 9.764168 GB avg = 0.163 KB (by entry num) total key len = 1.648884 GB avg = 0.028 KB total ukey len = 1.146494 GB avg = 0.016 KB total ukey num = 0.072981247 Billion total entry num = 0.059788976 Billion write speed all = 0.348929075 MB/sec (with seq num) write speed all = 0.334305738 MB/sec (without seq num) 2024/11/23-18:00:21.060411 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273664: 928520 keys, 36814600 bytes, temperature: kWarm 2024/11/23-18:00:21.060498 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356021060448, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273664, "file_size": 36814600, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 4012776021, "table_properties": {"data_size": 30480752, "index_size": 1766144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25998560, "raw_average_key_size": 28, "raw_value_size": 108219308, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928520, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356017, "oldest_key_time": 0, "file_creation_time": 1732356017, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273664, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:21.103221 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273665.sst) /dev/shm/ToplingTemp/Topling-141995-N0M3Ec 2024/11/23-18:00:21.988630 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273665.sst): first pass time = 0.89's, 151.595'MB/sec 2024/11/23-18:00:23.309746 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273665.sst): old prealloc_size = 147639500, real_size = 34343440 2024/11/23-18:00:23.310345 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273665.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928025 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0323 GB avg-val = 116.63 avg-zval = 34.84 2024/11/23-18:00:23.312178 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273665.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:24.134183 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273665.sst): second pass time = 1.32's, 81.977'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 46.263'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3080.204'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 290.841'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32692.951'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.977'MB/sec zip pipeline throughput = 81.977'MB/sec entries = 928025 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.63 avg-zval = 34.84 usrkeys = 928025 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.25 avg-zval = 34.84 TagRS{ kinds = RS_Key0_TagN bytes = 145120 } TagArray{ size = 745028 bytes = 2793872 } seq expand size = 5215196 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0323 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4455 value = 3.3473 dict = 1.00 all = 3.5191 } Zip/UnZip{ index = 0.0957 value = 0.2987 dict = 1.00 all = 0.2842 } ---------------------------- total value len = 9.877616 GB avg = 0.163 KB (by entry num) total key len = 1.674869 GB avg = 0.028 KB total ukey len = 1.165055 GB avg = 0.016 KB total ukey num = 0.073909272 Billion total entry num = 0.060717001 Billion write speed all = 0.353158258 MB/sec (with seq num) write speed all = 0.338309358 MB/sec (without seq num) 2024/11/23-18:00:24.181081 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273665: 928025 keys, 38139700 bytes, temperature: kWarm 2024/11/23-18:00:24.181154 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356024181119, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273665, "file_size": 38139700, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776210, "table_properties": {"data_size": 32334336, "index_size": 1776896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25984700, "raw_average_key_size": 28, "raw_value_size": 108233074, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928025, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356021, "oldest_key_time": 0, "file_creation_time": 1732356021, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273665, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:24.219131 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273666.sst) /dev/shm/ToplingTemp/Topling-141995-VE9zae 2024/11/23-18:00:25.211060 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273666.sst): first pass time = 0.99's, 135.315'MB/sec 2024/11/23-18:00:26.502807 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273666.sst): old prealloc_size = 147639500, real_size = 31503184 2024/11/23-18:00:26.503416 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273666.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933457 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0295 GB avg-val = 115.79 avg-zval = 31.60 2024/11/23-18:00:26.505391 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273666.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:27.350848 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273666.sst): second pass time = 1.29's, 83.721'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 42.142'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3030.775'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.325'MB/sec write SST data time = 0.13's, 276.576'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 26312.991'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.721'MB/sec zip pipeline throughput = 83.721'MB/sec entries = 933457 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.79 avg-zval = 31.60 usrkeys = 933457 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.79 avg-zval = 31.60 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933457 bytes = 3383792 } seq expand size = 6534199 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0357 GB } UnZip/Zip{ index = 10.5142 value = 3.6645 dict = 1.00 all = 3.7557 } Zip/UnZip{ index = 0.0951 value = 0.2729 dict = 1.00 all = 0.2663 } ---------------------------- total value len = 9.992231 GB avg = 0.162 KB (by entry num) total key len = 1.701005 GB avg = 0.028 KB total ukey len = 1.183724 GB avg = 0.016 KB total ukey num = 0.074842729 Billion total entry num = 0.061650458 Billion write speed all = 0.357425887 MB/sec (with seq num) write speed all = 0.342350184 MB/sec (without seq num) 2024/11/23-18:00:27.395201 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273666: 933457 keys, 35737410 bytes, temperature: kWarm 2024/11/23-18:00:27.395279 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356027395234, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273666, "file_size": 35737410, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3480997236, "largest_seqno": 4012776228, "table_properties": {"data_size": 29494016, "index_size": 1775616, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26136796, "raw_average_key_size": 28, "raw_value_size": 108081050, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933457, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356024, "oldest_key_time": 0, "file_creation_time": 1732356024, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273666, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:27.434666 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273667.sst) /dev/shm/ToplingTemp/Topling-141995-sFVPpd 2024/11/23-18:00:28.272413 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273667.sst): first pass time = 0.84's, 160.220'MB/sec 2024/11/23-18:00:29.584352 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273667.sst): old prealloc_size = 147639500, real_size = 33219488 2024/11/23-18:00:29.584915 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273667.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 931033 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1081 GB zip-val = 0.0312 GB avg-val = 116.16 avg-zval = 33.52 2024/11/23-18:00:29.586744 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273667.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:30.435114 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273667.sst): second pass time = 1.31's, 82.512'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.016'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3287.735'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.13's, 290.431'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 28594.835'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.512'MB/sec zip pipeline throughput = 82.512'MB/sec entries = 931033 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.16 avg-zval = 33.52 usrkeys = 931033 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.16 avg-zval = 33.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 931033 bytes = 3375008 } seq expand size = 6517231 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.4455 value = 3.4659 dict = 1.00 all = 3.5843 } Zip/UnZip{ index = 0.0957 value = 0.2885 dict = 1.00 all = 0.2790 } ---------------------------- total value len = 10.106897 GB avg = 0.161 KB (by entry num) total key len = 1.727074 GB avg = 0.028 KB total ukey len = 1.202345 GB avg = 0.016 KB total ukey num = 0.075773762 Billion total entry num = 0.062581491 Billion write speed all = 0.361693622 MB/sec (with seq num) write speed all = 0.346391691 MB/sec (without seq num) 2024/11/23-18:00:30.481062 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273667: 931033 keys, 37446172 bytes, temperature: kWarm 2024/11/23-18:00:30.481142 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356030481099, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273667, "file_size": 37446172, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3512539342, "largest_seqno": 4012776268, "table_properties": {"data_size": 31203920, "index_size": 1782656, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26068924, "raw_average_key_size": 28, "raw_value_size": 108148938, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 931033, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356027, "oldest_key_time": 0, "file_creation_time": 1732356027, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273667, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:30.525193 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273668.sst) /dev/shm/ToplingTemp/Topling-141995-hntf9c 2024/11/23-18:00:31.541611 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273668.sst): first pass time = 1.02's, 132.055'MB/sec 2024/11/23-18:00:32.888586 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273668.sst): old prealloc_size = 147639500, real_size = 34596048 2024/11/23-18:00:32.889172 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273668.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924572 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0326 GB avg-val = 117.17 avg-zval = 35.25 2024/11/23-18:00:32.890987 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273668.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:33.737576 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273668.sst): second pass time = 1.35's, 80.475'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 45.843'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3149.215'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.320'MB/sec write SST data time = 0.13's, 308.073'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34361.983'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.475'MB/sec zip pipeline throughput = 80.475'MB/sec entries = 924572 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.17 avg-zval = 35.25 usrkeys = 924572 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.17 avg-zval = 35.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924572 bytes = 3467152 } seq expand size = 6472004 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0326 GB dict = 1.08 MB all = 0.0389 GB } UnZip/Zip{ index = 10.4423 value = 3.3236 dict = 1.00 all = 3.4486 } Zip/UnZip{ index = 0.0958 value = 0.3009 dict = 1.00 all = 0.2900 } ---------------------------- total value len = 10.221699 GB avg = 0.161 KB (by entry num) total key len = 1.752962 GB avg = 0.028 KB total ukey len = 1.220836 GB avg = 0.016 KB total ukey num = 0.076698334 Billion total entry num = 0.063506063 Billion write speed all = 0.365956727 MB/sec (with seq num) write speed all = 0.350430295 MB/sec (without seq num) 2024/11/23-18:00:33.785302 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273668: 924572 keys, 38918996 bytes, temperature: kWarm 2024/11/23-18:00:33.785374 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356033785337, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273668, "file_size": 38918996, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3411274482, "largest_seqno": 4012774796, "table_properties": {"data_size": 32593920, "index_size": 1770816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25888016, "raw_average_key_size": 28, "raw_value_size": 108329768, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924572, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356030, "oldest_key_time": 0, "file_creation_time": 1732356030, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273668, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:33.821441 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273669.sst) /dev/shm/ToplingTemp/Topling-141995-au5Wte 2024/11/23-18:00:34.666848 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273669.sst): first pass time = 0.85's, 158.768'MB/sec 2024/11/23-18:00:35.977308 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273669.sst): old prealloc_size = 147639500, real_size = 32561072 2024/11/23-18:00:35.977853 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273669.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933794 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0306 GB avg-val = 115.73 avg-zval = 32.72 2024/11/23-18:00:35.979701 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273669.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:36.839486 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273669.sst): second pass time = 1.31's, 82.525'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.680'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3412.935'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 287.599'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30985.662'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.525'MB/sec zip pipeline throughput = 82.525'MB/sec entries = 933794 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.73 avg-zval = 32.72 usrkeys = 933794 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.73 avg-zval = 32.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933794 bytes = 3385024 } seq expand size = 6536558 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0368 GB } UnZip/Zip{ index = 10.5138 value = 3.5374 dict = 1.00 all = 3.6475 } Zip/UnZip{ index = 0.0951 value = 0.2827 dict = 1.00 all = 0.2742 } ---------------------------- total value len = 10.336307 GB avg = 0.160 KB (by entry num) total key len = 1.779108 GB avg = 0.028 KB total ukey len = 1.239512 GB avg = 0.016 KB total ukey num = 0.077632128 Billion total entry num = 0.064439857 Billion write speed all = 0.370223216 MB/sec (with seq num) write speed all = 0.354469977 MB/sec (without seq num) 2024/11/23-18:00:36.887308 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273669: 933794 keys, 36797002 bytes, temperature: kWarm 2024/11/23-18:00:36.887381 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356036887344, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273669, "file_size": 36797002, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3525889795, "largest_seqno": 4012776316, "table_properties": {"data_size": 30551136, "index_size": 1776320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26146232, "raw_average_key_size": 28, "raw_value_size": 108071628, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933794, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356033, "oldest_key_time": 0, "file_creation_time": 1732356033, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273669, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:36.925514 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273670.sst) /dev/shm/ToplingTemp/Topling-141995-y50zze 2024/11/23-18:00:37.842429 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273670.sst): first pass time = 0.92's, 146.386'MB/sec 2024/11/23-18:00:39.152581 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273670.sst): old prealloc_size = 147639500, real_size = 33583504 2024/11/23-18:00:39.153149 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273670.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923213 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.38 avg-zval = 34.22 2024/11/23-18:00:39.154969 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273670.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:40.008836 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273670.sst): second pass time = 1.31's, 82.765'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 44.268'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3223.646'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.316'MB/sec write SST data time = 0.13's, 300.754'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34821.816'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.765'MB/sec zip pipeline throughput = 82.765'MB/sec entries = 923213 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.38 avg-zval = 34.22 usrkeys = 923213 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.38 avg-zval = 34.22 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923213 bytes = 3462064 } seq expand size = 6462491 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5128 value = 3.4298 dict = 1.00 all = 3.5410 } Zip/UnZip{ index = 0.0951 value = 0.2916 dict = 1.00 all = 0.2824 } ---------------------------- total value len = 10.451138 GB avg = 0.160 KB (by entry num) total key len = 1.804958 GB avg = 0.028 KB total ukey len = 1.257976 GB avg = 0.016 KB total ukey num = 0.078555341 Billion total entry num = 0.065363070 Billion write speed all = 0.374485860 MB/sec (with seq num) write speed all = 0.358508475 MB/sec (without seq num) 2024/11/23-18:00:40.056042 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273670: 923213 keys, 37903414 bytes, temperature: kWarm 2024/11/23-18:00:40.056117 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356040056079, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273670, "file_size": 37903414, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 4012776344, "table_properties": {"data_size": 31596160, "index_size": 1756352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25849964, "raw_average_key_size": 28, "raw_value_size": 108367830, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923213, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356036, "oldest_key_time": 0, "file_creation_time": 1732356036, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273670, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:40.095150 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273671.sst) /dev/shm/ToplingTemp/Topling-141995-Dpoocc 2024/11/23-18:00:41.128124 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273671.sst): first pass time = 1.03's, 129.938'MB/sec 2024/11/23-18:00:42.432861 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273671.sst): old prealloc_size = 147639500, real_size = 32920912 2024/11/23-18:00:42.433448 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273671.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928015 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.63 avg-zval = 33.30 2024/11/23-18:00:42.434284 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273671.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:43.290506 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273671.sst): second pass time = 1.30's, 83.003'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 43.439'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3190.526'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.317'MB/sec write SST data time = 0.13's, 296.124'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 32420.616'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.003'MB/sec zip pipeline throughput = 83.003'MB/sec entries = 928015 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.63 avg-zval = 33.30 usrkeys = 928015 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.63 avg-zval = 33.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928015 bytes = 3480064 } seq expand size = 6496105 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.3755 value = 3.5027 dict = 1.00 all = 3.6028 } Zip/UnZip{ index = 0.0964 value = 0.2855 dict = 1.00 all = 0.2776 } ---------------------------- total value len = 10.565867 GB avg = 0.159 KB (by entry num) total key len = 1.830943 GB avg = 0.028 KB total ukey len = 1.276536 GB avg = 0.016 KB total ukey num = 0.079483356 Billion total entry num = 0.066291085 Billion write speed all = 0.378747403 MB/sec (with seq num) write speed all = 0.362544799 MB/sec (without seq num) 2024/11/23-18:00:43.336757 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273671: 928015 keys, 37254148 bytes, temperature: kWarm 2024/11/23-18:00:43.336832 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356043336796, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273671, "file_size": 37254148, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3471845708, "largest_seqno": 4012771944, "table_properties": {"data_size": 30899840, "index_size": 1788864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25984420, "raw_average_key_size": 28, "raw_value_size": 108233318, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928015, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356040, "oldest_key_time": 0, "file_creation_time": 1732356040, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273671, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:43.378302 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273672.sst) /dev/shm/ToplingTemp/Topling-141995-ycmNCd 2024/11/23-18:00:44.244750 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273672.sst): first pass time = 0.87's, 154.912'MB/sec 2024/11/23-18:00:45.683775 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273672.sst): old prealloc_size = 147639500, real_size = 35498608 2024/11/23-18:00:45.684401 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273672.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929559 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.39 avg-zval = 36.02 2024/11/23-18:00:45.684792 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273672.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:46.446140 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273672.sst): second pass time = 1.44's, 75.222'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.76's, 49.508'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2949.856'MB/sec (index lex order gen) rebuild zvType time = 0.64's, 0.365'MB/sec write SST data time = 0.13's, 300.789'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28484.986'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.222'MB/sec zip pipeline throughput = 75.222'MB/sec entries = 929559 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.39 avg-zval = 36.02 usrkeys = 929559 avg-key = 20.00 avg-zkey = 1.92 avg-val = 119.26 avg-zval = 36.02 TagRS{ kinds = RS_Key0_TagN bytes = 145360 } TagArray{ size = 381564 bytes = 1240096 } seq expand size = 2670948 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.4428 value = 3.2309 dict = 1.00 all = 3.5562 } Zip/UnZip{ index = 0.0958 value = 0.3095 dict = 1.00 all = 0.2812 } ---------------------------- total value len = 10.676728 GB avg = 0.159 KB (by entry num) total key len = 1.856971 GB avg = 0.028 KB total ukey len = 1.295128 GB avg = 0.016 KB total ukey num = 0.080412915 Billion total entry num = 0.067220644 Billion write speed all = 0.382892712 MB/sec (with seq num) write speed all = 0.366464492 MB/sec (without seq num) 2024/11/23-18:00:46.492813 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273672: 929559 keys, 37741747 bytes, temperature: kWarm 2024/11/23-18:00:46.492889 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356046492852, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273672, "file_size": 37741747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4012776360, "table_properties": {"data_size": 33485728, "index_size": 1780288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26027652, "raw_average_key_size": 28, "raw_value_size": 108190094, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929559, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356043, "oldest_key_time": 0, "file_creation_time": 1732356043, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273672, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:46.530340 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273673.sst) /dev/shm/ToplingTemp/Topling-141995-xlC6md 2024/11/23-18:00:47.347470 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273673.sst): first pass time = 0.82's, 164.262'MB/sec 2024/11/23-18:00:48.834961 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273673.sst): old prealloc_size = 147639500, real_size = 37003680 2024/11/23-18:00:48.835541 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273673.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926553 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0350 GB avg-val = 116.86 avg-zval = 37.77 2024/11/23-18:00:48.835817 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273673.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:49.548176 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273673.sst): second pass time = 1.49's, 72.830'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.71's, 54.200'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3342.557'MB/sec (index lex order gen) rebuild zvType time = 0.59's, 0.393'MB/sec write SST data time = 0.12's, 313.881'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33191.753'MB/sec zip my value time = 1.49's, unzip length = 0.108'GB zip my value throughput = 72.830'MB/sec zip pipeline throughput = 72.830'MB/sec entries = 926553 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.86 avg-zval = 37.77 usrkeys = 926553 avg-key = 20.00 avg-zkey = 1.91 avg-val = 118.16 avg-zval = 37.77 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 172955 bytes = 648592 } seq expand size = 1210685 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0350 GB dict = 1.08 MB all = 0.0387 GB } UnZip/Zip{ index = 10.4462 value = 3.0937 dict = 1.00 all = 3.4722 } Zip/UnZip{ index = 0.0957 value = 0.3232 dict = 1.00 all = 0.2880 } ---------------------------- total value len = 10.786213 GB avg = 0.158 KB (by entry num) total key len = 1.882914 GB avg = 0.028 KB total ukey len = 1.313659 GB avg = 0.016 KB total ukey num = 0.081339468 Billion total entry num = 0.068147197 Billion write speed all = 0.386993254 MB/sec (with seq num) write speed all = 0.370340169 MB/sec (without seq num) 2024/11/23-18:00:49.595627 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273673: 926553 keys, 38655283 bytes, temperature: kWarm 2024/11/23-18:00:49.595701 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356049595664, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273673, "file_size": 38655283, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4016317816, "table_properties": {"data_size": 34997904, "index_size": 1773952, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25943484, "raw_average_key_size": 28, "raw_value_size": 108274370, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926553, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356046, "oldest_key_time": 0, "file_creation_time": 1732356046, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273673, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:49.633154 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273674.sst) /dev/shm/ToplingTemp/Topling-141995-mu27Cd 2024/11/23-18:00:50.688571 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273674.sst): first pass time = 1.06's, 127.175'MB/sec 2024/11/23-18:00:52.041581 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273674.sst): old prealloc_size = 147639500, real_size = 35382288 2024/11/23-18:00:52.042154 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273674.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928109 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0334 GB avg-val = 116.61 avg-zval = 35.97 2024/11/23-18:00:52.044084 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273674.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:52.878119 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273674.sst): second pass time = 1.35's, 80.037'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 47.202'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3229.007'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.327'MB/sec write SST data time = 0.13's, 311.022'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34296.764'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.037'MB/sec zip pipeline throughput = 80.037'MB/sec entries = 928109 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.61 avg-zval = 35.97 usrkeys = 928109 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.61 avg-zval = 35.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928109 bytes = 3248400 } seq expand size = 6496763 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0334 GB dict = 1.08 MB all = 0.0395 GB } UnZip/Zip{ index = 10.5142 value = 3.2419 dict = 1.00 all = 3.3992 } Zip/UnZip{ index = 0.0951 value = 0.3085 dict = 1.00 all = 0.2942 } ---------------------------- total value len = 10.900941 GB avg = 0.158 KB (by entry num) total key len = 1.908901 GB avg = 0.028 KB total ukey len = 1.332221 GB avg = 0.016 KB total ukey num = 0.082267577 Billion total entry num = 0.069075306 Billion write speed all = 0.391251748 MB/sec (with seq num) write speed all = 0.374373579 MB/sec (without seq num) 2024/11/23-18:00:52.926070 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273674: 928109 keys, 39484604 bytes, temperature: kWarm 2024/11/23-18:00:52.926142 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356052926106, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273674, "file_size": 39484604, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4016317817, "largest_seqno": 4166410134, "table_properties": {"data_size": 33384640, "index_size": 1765440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25987052, "raw_average_key_size": 28, "raw_value_size": 108230782, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928109, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356049, "oldest_key_time": 0, "file_creation_time": 1732356049, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273674, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:52.966068 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273675.sst) /dev/shm/ToplingTemp/Topling-141995-9NvUGb 2024/11/23-18:00:53.864569 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273675.sst): first pass time = 0.90's, 149.387'MB/sec 2024/11/23-18:00:55.176131 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273675.sst): old prealloc_size = 147639500, real_size = 35262688 2024/11/23-18:00:55.176719 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273675.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926421 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0333 GB avg-val = 116.88 avg-zval = 35.91 2024/11/23-18:00:55.177462 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273675.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:56.014526 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273675.sst): second pass time = 1.31's, 82.608'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 46.672'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3121.572'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.326'MB/sec write SST data time = 0.13's, 309.191'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34175.274'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.608'MB/sec zip pipeline throughput = 82.608'MB/sec entries = 926421 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.88 avg-zval = 35.91 usrkeys = 926421 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.88 avg-zval = 35.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926421 bytes = 3010880 } seq expand size = 6484947 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0333 GB dict = 1.08 MB all = 0.0391 GB } UnZip/Zip{ index = 10.5134 value = 3.2547 dict = 1.00 all = 3.4302 } Zip/UnZip{ index = 0.0951 value = 0.3073 dict = 1.00 all = 0.2915 } ---------------------------- total value len = 11.015704 GB avg = 0.157 KB (by entry num) total key len = 1.934841 GB avg = 0.028 KB total ukey len = 1.350749 GB avg = 0.016 KB total ukey num = 0.083193998 Billion total entry num = 0.070001727 Billion write speed all = 0.395511351 MB/sec (with seq num) write speed all = 0.378408455 MB/sec (without seq num) 2024/11/23-18:00:56.062468 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273675: 926421 keys, 39128412 bytes, temperature: kWarm 2024/11/23-18:00:56.062541 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356056062505, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273675, "file_size": 39128412, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4116739076, "largest_seqno": 4166409769, "table_properties": {"data_size": 33268560, "index_size": 1762368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25939788, "raw_average_key_size": 28, "raw_value_size": 108277982, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926421, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356052, "oldest_key_time": 0, "file_creation_time": 1732356052, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273675, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:56.101362 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273676.sst) /dev/shm/ToplingTemp/Topling-141995-mijHWb 2024/11/23-18:00:56.964854 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273676.sst): first pass time = 0.86's, 155.442'MB/sec 2024/11/23-18:00:58.322564 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273676.sst): old prealloc_size = 147639500, real_size = 34236288 2024/11/23-18:00:58.323183 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273676.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927469 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0322 GB avg-val = 116.71 avg-zval = 34.76 2024/11/23-18:00:58.323970 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273676.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-18:00:59.176385 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273676.sst): second pass time = 1.36's, 79.776'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 44.770'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2976.418'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 298.477'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31910.133'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.776'MB/sec zip pipeline throughput = 79.776'MB/sec entries = 927469 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.71 avg-zval = 34.76 usrkeys = 927469 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.71 avg-zval = 34.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927469 bytes = 3130224 } seq expand size = 6492283 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0322 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.5119 value = 3.3576 dict = 1.00 all = 3.5113 } Zip/UnZip{ index = 0.0951 value = 0.2978 dict = 1.00 all = 0.2848 } ---------------------------- total value len = 11.130445 GB avg = 0.157 KB (by entry num) total key len = 1.960810 GB avg = 0.028 KB total ukey len = 1.369299 GB avg = 0.016 KB total ukey num = 0.084121467 Billion total entry num = 0.070929196 Billion write speed all = 0.399770052 MB/sec (with seq num) write speed all = 0.382442228 MB/sec (without seq num) 2024/11/23-18:00:59.223360 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273676: 927469 keys, 38224174 bytes, temperature: kWarm 2024/11/23-18:00:59.223432 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356059223396, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273676, "file_size": 38224174, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4050411210, "largest_seqno": 4166409877, "table_properties": {"data_size": 32239600, "index_size": 1764608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25969132, "raw_average_key_size": 28, "raw_value_size": 108248634, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927469, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356056, "oldest_key_time": 0, "file_creation_time": 1732356056, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273676, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:59.313310 139957265520192 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273677.sst) /dev/shm/ToplingTemp/Topling-141995-qc1sob 2024/11/23-18:00:59.447795 139957265520192 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273677.sst): first pass time = 0.13's, 235.135'MB/sec 2024/11/23-18:00:59.741947 139957265520192 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273677.sst): old prealloc_size = 147639500, real_size = 8485184 2024/11/23-18:00:59.742172 139957265520192 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273677.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 220430 prefix = 4 raw-key = 0.0035 GB zip-key = 0.0004 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0254 GB zip-val = 0.0080 GB avg-val = 115.40 avg-zval = 36.31 2024/11/23-18:00:59.742390 139957265520192 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273677.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0005 GB, waited 0.000 sec, Key+Value = 0.032 GB 2024/11/23-18:00:59.940516 139957265520192 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00/273677.sst): second pass time = 0.29's, 86.697'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.20's, 47.261'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2440.404'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.330'MB/sec write SST data time = 0.03's, 297.209'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.263'MB, throughput = 8359.758'MB/sec zip my value time = 0.29's, unzip length = 0.025'GB zip my value throughput = 86.697'MB/sec zip pipeline throughput = 86.697'MB/sec entries = 220430 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.40 avg-zval = 36.31 usrkeys = 220430 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.40 avg-zval = 36.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 220430 bytes = 688864 } seq expand size = 1543010 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0044 GB value = 0.0254 GB dict = 0.26 MB all = 0.0316 GB } __ZipSize{ index = 0.0004 GB value = 0.0080 GB dict = 0.26 MB all = 0.0094 GB } UnZip/Zip{ index = 10.3539 value = 3.1780 dict = 1.00 all = 3.3685 } Zip/UnZip{ index = 0.0966 value = 0.3147 dict = 1.00 all = 0.2969 } ---------------------------- total value len = 11.157425 GB avg = 0.157 KB (by entry num) total key len = 1.966982 GB avg = 0.028 KB total ukey len = 1.373707 GB avg = 0.016 KB total ukey num = 0.084341897 Billion total entry num = 0.071149626 Billion write speed all = 0.400773067 MB/sec (with seq num) write speed all = 0.383391799 MB/sec (without seq num) 2024/11/23-18:00:59.957541 139957265520192 [compaction_job.cc:1948] [default] [JOB 32564] Generated table #273677: 220430 keys, 9383781 bytes, temperature: kWarm 2024/11/23-18:00:59.957612 139957265520192 EVENT_LOG_v1 {"time_micros": 1732356059957579, "cf_name": "default", "job": 32564, "event": "table_file_creation", "file_number": 273677, "file_size": 9383781, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4137892167, "largest_seqno": 4166408987, "table_properties": {"data_size": 8004080, "index_size": 425792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 6172040, "raw_average_key_size": 28, "raw_value_size": 25436900, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 220430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732356059, "oldest_key_time": 0, "file_creation_time": 1732356059, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 273677, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-18:00:59.970889 139957265520192 [compaction_job.cc:703] job-32564: subcompact[0], size: 0.771530 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-32564/att-00 [273657,273658,273659,273660,273661,273662,273663,273664,273665,273666,273667,273668,273669,273670,273671,273672,273673,273674,273675,273676,273677] 2024/11/23-18:01:00.098275 139957265520192 (Original Log Time 2024/11/23-17:59:52.381230) EVENT_LOG_v1 {"time_micros": 1732355992381166, "job": 32564, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [273621], "files_L3": [273643, 273644, 273569, 273570, 273571, 273572, 273573, 273574, 273575, 273576, 273577, 273578, 273579, 273580, 273581, 273582, 273583, 273584, 273589, 273590, 273649], "score": 0.992637, "input_data_size": 791903280, "oldest_snapshot_seqno": 4166734211} 2024/11/23-18:01:00.098279 139957265520192 (Original Log Time 2024/11/23-18:01:00.096570) [compaction_job.cc:2022] [default] [JOB 32564] Compacted 1@2 + 21@3 files to L3 => 771529997 bytes 2024/11/23-18:01:00.098280 139957265520192 (Original Log Time 2024/11/23-18:01:00.098192) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 11.7 rd, 11.4 wr, level 3, files in(1, 21) out(21 +0 blob) MB in(27.1, 728.1 +0.0 blob) out(735.8 +0.0 blob), read-write-amplify(55.0) write-amplify(27.1) OK, records in: 18878602, records dropped: 87140 output_compression: Snappy 2024/11/23-18:01:00.098284 139957265520192 (Original Log Time 2024/11/23-18:01:00.098236) EVENT_LOG_v1 {"time_micros": 1732356060098215, "cf": "default", "job": 32564, "event": "compaction_finished", "compaction_time_micros": 67589666, "compaction_time_cpu_micros": 28658467, "output_level": 3, "num_output_files": 21, "total_output_size": 771529997, "num_input_records": 18878602, "num_output_records": 18791462, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 561337569, "file_range_sync_nanos": 0, "file_fsync_nanos": 15345, "file_prepare_write_nanos": 64315, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/23-18:01:00.236817 139957265520192 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732355992,"executesMs":67887,"compactionJobId":32564,"attempt":0,"compactionInputRawBytes":2728338980,"compactionInputZipBytes":791903280,"compactionOutputRawBytes":0,"compactionOutputZipBytes":771529997,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-66bb56e5f7a6f799ff75c966396bbadc-2ecbaa89b10ac56d-00"}}, response = 2024/11/23-18:01:00.236913 139957265520192 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-32564/att-00: olev 3, work 67.887 s, result 7.823 ms, install 1.730 ms, input{raw 2.541 GiB zip 755.218 MiB}