2024/11/23-14:21:42.547747 139957475337792 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00: fork to child time = 0.010539 sec 2024/11/23-14:21:42.548369 139957475337792 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00 2024/11/23-14:21:42.556991 139957475337792 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00 2024/11/23-14:21:42.557853 139957475337792 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:21:42.558095 139957475337792 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:21:42.615145 139957475337792 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27621/att-00/MANIFEST-206347 2024/11/23-14:21:42.637939 139957475337792 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27621/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 262917, last_sequence is 4018400065, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 262690 2024/11/23-14:21:42.637948 139957475337792 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:21:42.638089 139957475337792 [version_set.cc:5721] Creating manifest 262917 2024/11/23-14:21:42.675858 139957475337792 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:21:42.697791 139957475337792 [compaction_job.cc:2437] [default] [JOB 27621] Compacting 1@2 + 19@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-14:21:42.697808 139957475337792 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 20666 Base level 2, inputs: [262912(31MB)], [262865(40MB) 262866(40MB) 262867(40MB) 262868(40MB) 262869(40MB) 262870(40MB) 262871(40MB) 262872(40MB) 262873(40MB) 262874(40MB) 262875(40MB) 262876(40MB) 262877(40MB) 262878(40MB) 262879(40MB) 262880(40MB) 262881(40MB) 262882(39MB) 262883(39MB)] 2024/11/23-14:21:42.705587 139957475337792 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:21:42.705617 139957475337792 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:21:42.705634 139957475337792 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:21:42.705640 139957475337792 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:21:42.705652 139957475337792 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:21:43.382630 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262918.sst) /dev/shm/ToplingTemp/Topling-71322-yaWL6f 2024/11/23-14:21:43.667017 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262918.sst): first pass time = 0.28's, 472.016'MB/sec 2024/11/23-14:21:44.719030 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262918.sst): old prealloc_size = 147639500, real_size = 40697728 2024/11/23-14:21:44.719163 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262918.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.99 avg-zval = 312.38 2024/11/23-14:21:44.719342 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262918.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:21:44.885653 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262918.sst): second pass time = 1.05's, 125.211'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 255.264'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2029.636'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.05's, 932.595'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 58152.243'MB/sec zip my value time = 1.05's, unzip length = 0.132'GB zip my value throughput = 125.211'MB/sec zip pipeline throughput = 125.211'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.99 avg-zval = 312.38 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.99 avg-zval = 312.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6507 value = 3.2556 dict = 1.00 all = 3.1558 } Zip/UnZip{ index = 0.1504 value = 0.3072 dict = 1.00 all = 0.3169 } ---------------------------- total value len = 8.981155 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679866 Billion total entry num = 0.052487595 Billion write speed all = 0.532064510 MB/sec (with seq num) write speed all = 0.510632988 MB/sec (without seq num) 2024/11/23-14:21:44.939252 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262918: 129431 keys, 42530661 bytes, temperature: kWarm 2024/11/23-14:21:44.939332 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342904939280, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262918, "file_size": 42530661, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4018345153, "table_properties": {"data_size": 40431664, "index_size": 233536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629396, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342903, "oldest_key_time": 0, "file_creation_time": 1732342903, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262918, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:21:44.978864 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262919.sst) /dev/shm/ToplingTemp/Topling-71322-gjsHEe 2024/11/23-14:21:45.612452 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262919.sst): first pass time = 0.63's, 211.850'MB/sec 2024/11/23-14:21:46.490257 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262919.sst): old prealloc_size = 147639500, real_size = 40810400 2024/11/23-14:21:46.490389 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262919.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.98 avg-zval = 313.29 2024/11/23-14:21:46.490549 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262919.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:21:46.655714 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262919.sst): second pass time = 0.88's, 150.068'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 257.758'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2098.226'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.05's, 922.423'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48507.324'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 150.068'MB/sec zip pipeline throughput = 150.068'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1016.98 avg-zval = 313.29 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1023.98 avg-zval = 313.29 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7826 value = 3.2462 dict = 1.00 all = 3.1475 } Zip/UnZip{ index = 0.1474 value = 0.3081 dict = 1.00 all = 0.3177 } ---------------------------- total value len = 9.113691 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809297 Billion total entry num = 0.052617026 Billion write speed all = 0.538912480 MB/sec (with seq num) write speed all = 0.517430050 MB/sec (without seq num) 2024/11/23-14:21:46.710616 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262919: 129431 keys, 42642759 bytes, temperature: kWarm 2024/11/23-14:21:46.710687 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342906710650, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262919, "file_size": 42642759, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4018344815, "table_properties": {"data_size": 40548880, "index_size": 228992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629294, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342904, "oldest_key_time": 0, "file_creation_time": 1732342904, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262919, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:21:46.754895 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262920.sst) /dev/shm/ToplingTemp/Topling-71322-uzUrqc 2024/11/23-14:21:47.281941 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262920.sst): first pass time = 0.53's, 254.682'MB/sec 2024/11/23-14:21:48.208426 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262920.sst): old prealloc_size = 147639500, real_size = 40732432 2024/11/23-14:21:48.208548 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262920.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.79 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.97 avg-zval = 312.65 2024/11/23-14:21:48.208726 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262920.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:21:48.695523 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262920.sst): second pass time = 0.93's, 142.185'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.49's, 87.394'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2340.723'MB/sec (index lex order gen) rebuild zvType time = 0.34's, 0.094'MB/sec write SST data time = 0.14's, 298.222'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48041.102'MB/sec zip my value time = 0.93's, unzip length = 0.132'GB zip my value throughput = 142.185'MB/sec zip pipeline throughput = 142.185'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.79 avg-val = 1016.97 avg-zval = 312.65 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.79 avg-val = 1023.97 avg-zval = 312.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.6930 value = 3.2527 dict = 1.00 all = 3.1532 } Zip/UnZip{ index = 0.1494 value = 0.3074 dict = 1.00 all = 0.3171 } ---------------------------- total value len = 9.246226 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938730 Billion total entry num = 0.052746459 Billion write speed all = 0.545751726 MB/sec (with seq num) write speed all = 0.524218693 MB/sec (without seq num) 2024/11/23-14:21:48.764966 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262920: 129433 keys, 42565258 bytes, temperature: kWarm 2024/11/23-14:21:48.765065 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342908765017, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262920, "file_size": 42565258, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237870807, "largest_seqno": 4018347536, "table_properties": {"data_size": 40467840, "index_size": 232064, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629723, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342906, "oldest_key_time": 0, "file_creation_time": 1732342906, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262920, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:21:48.841385 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262921.sst) /dev/shm/ToplingTemp/Topling-71322-4o7UWc 2024/11/23-14:21:49.629466 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262921.sst): first pass time = 0.79's, 170.321'MB/sec 2024/11/23-14:21:50.836536 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262921.sst): old prealloc_size = 147639500, real_size = 40775840 2024/11/23-14:21:50.836658 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262921.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.78 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.96 avg-zval = 313.00 2024/11/23-14:21:50.836819 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262921.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:21:51.028533 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262921.sst): second pass time = 1.21's, 109.103'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 221.940'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2273.060'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.230'MB/sec write SST data time = 0.05's, 835.144'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44324.157'MB/sec zip my value time = 1.21's, unzip length = 0.132'GB zip my value throughput = 109.103'MB/sec zip pipeline throughput = 153.172'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.78 avg-val = 1016.96 avg-zval = 313.00 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.78 avg-val = 1023.96 avg-zval = 313.00 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129434 bytes = 485392 } seq expand size = 906038 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7432 value = 3.2491 dict = 1.00 all = 3.1500 } Zip/UnZip{ index = 0.1483 value = 0.3078 dict = 1.00 all = 0.3175 } ---------------------------- total value len = 9.378762 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068164 Billion total entry num = 0.052875893 Billion write speed all = 0.552581270 MB/sec (with seq num) write speed all = 0.530997967 MB/sec (without seq num) 2024/11/23-14:21:51.082796 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262921: 129434 keys, 42608762 bytes, temperature: kWarm 2024/11/23-14:21:51.082857 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342911082825, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262921, "file_size": 42608762, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4018344758, "table_properties": {"data_size": 40512976, "index_size": 230336, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629511, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342908, "oldest_key_time": 0, "file_creation_time": 1732342908, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262921, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:21:51.127210 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262922.sst) /dev/shm/ToplingTemp/Topling-71322-I3ZcTd 2024/11/23-14:21:51.692575 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262922.sst): first pass time = 0.57's, 237.422'MB/sec 2024/11/23-14:21:52.570720 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262922.sst): old prealloc_size = 147639500, real_size = 40593888 2024/11/23-14:21:52.570841 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262922.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.83 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.56 2024/11/23-14:21:52.571001 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262922.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:21:52.740290 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262922.sst): second pass time = 0.88's, 150.196'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 250.232'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2364.589'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.05's, 861.690'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48318.794'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 150.196'MB/sec zip pipeline throughput = 151.580'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.83 avg-val = 1017.00 avg-zval = 311.56 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.83 avg-val = 1024.00 avg-zval = 311.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5661 value = 3.2642 dict = 1.00 all = 3.1635 } Zip/UnZip{ index = 0.1523 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.511298 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197594 Billion total entry num = 0.053005323 Billion write speed all = 0.559426930 MB/sec (with seq num) write speed all = 0.537792685 MB/sec (without seq num) 2024/11/23-14:21:52.792202 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262922: 129430 keys, 42426691 bytes, temperature: kWarm 2024/11/23-14:21:52.792271 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342912792234, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262922, "file_size": 42426691, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4018350464, "table_properties": {"data_size": 40324816, "index_size": 236544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629805, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342911, "oldest_key_time": 0, "file_creation_time": 1732342911, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262922, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:21:52.835600 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262923.sst) /dev/shm/ToplingTemp/Topling-71322-yItuse 2024/11/23-14:21:53.943881 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262923.sst): first pass time = 1.11's, 121.110'MB/sec 2024/11/23-14:21:55.247913 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262923.sst): old prealloc_size = 147639500, real_size = 41005968 2024/11/23-14:21:55.248082 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262923.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.01 avg-zval = 314.83 2024/11/23-14:21:55.248268 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262923.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:21:55.439029 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262923.sst): second pass time = 1.29's, 102.086'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 224.184'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1545.995'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 0.222'MB/sec write SST data time = 0.05's, 951.241'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47839.725'MB/sec zip my value time = 1.29's, unzip length = 0.132'GB zip my value throughput = 102.086'MB/sec zip pipeline throughput = 129.754'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1017.01 avg-zval = 314.83 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1024.01 avg-zval = 314.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129428 bytes = 485376 } seq expand size = 905996 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.8767 value = 3.2304 dict = 1.00 all = 3.1331 } Zip/UnZip{ index = 0.1454 value = 0.3096 dict = 1.00 all = 0.3192 } ---------------------------- total value len = 9.643833 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327022 Billion total entry num = 0.053134751 Billion write speed all = 0.566242872 MB/sec (with seq num) write speed all = 0.544558786 MB/sec (without seq num) 2024/11/23-14:21:55.494427 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262923: 129428 keys, 42839141 bytes, temperature: kWarm 2024/11/23-14:21:55.494497 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342915494461, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262923, "file_size": 42839141, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237902993, "largest_seqno": 4018347434, "table_properties": {"data_size": 40747584, "index_size": 225856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629624, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342912, "oldest_key_time": 0, "file_creation_time": 1732342912, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262923, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:21:55.532492 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262924.sst) /dev/shm/ToplingTemp/Topling-71322-gKGRPf 2024/11/23-14:21:56.070346 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262924.sst): first pass time = 0.54's, 249.561'MB/sec 2024/11/23-14:21:56.947531 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262924.sst): old prealloc_size = 147639500, real_size = 40964832 2024/11/23-14:21:56.947643 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262924.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.01 avg-zval = 314.53 2024/11/23-14:21:56.947794 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262924.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:21:57.110938 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262924.sst): second pass time = 0.88's, 150.311'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.940'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2416.789'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 952.684'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 50010.707'MB/sec zip my value time = 0.88's, unzip length = 0.132'GB zip my value throughput = 150.311'MB/sec zip pipeline throughput = 150.311'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.01 avg-zval = 314.53 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.01 avg-zval = 314.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129428 bytes = 485376 } seq expand size = 905996 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9376 value = 3.2335 dict = 1.00 all = 3.1361 } Zip/UnZip{ index = 0.1441 value = 0.3093 dict = 1.00 all = 0.3189 } ---------------------------- total value len = 9.776369 GB avg = 0.184 KB (by entry num) total key len = 1.458941 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456450 Billion total entry num = 0.053264179 Billion write speed all = 0.573086933 MB/sec (with seq num) write speed all = 0.551351882 MB/sec (without seq num) 2024/11/23-14:21:57.163731 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262924: 129428 keys, 42797478 bytes, temperature: kWarm 2024/11/23-14:21:57.163801 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342917163763, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262924, "file_size": 42797478, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4018343422, "table_properties": {"data_size": 40708432, "index_size": 223872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629259, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342915, "oldest_key_time": 0, "file_creation_time": 1732342915, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262924, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:21:57.205706 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262925.sst) /dev/shm/ToplingTemp/Topling-71322-8BVQRg 2024/11/23-14:21:58.295650 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262925.sst): first pass time = 1.09's, 123.146'MB/sec 2024/11/23-14:21:59.425141 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262925.sst): old prealloc_size = 147639500, real_size = 40763232 2024/11/23-14:21:59.425301 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262925.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.99 avg-zval = 312.92 2024/11/23-14:21:59.425517 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262925.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:21:59.662747 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262925.sst): second pass time = 1.12's, 118.008'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.24's, 179.285'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1708.211'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.182'MB/sec write SST data time = 0.06's, 714.836'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46129.636'MB/sec zip my value time = 1.12's, unzip length = 0.132'GB zip my value throughput = 118.008'MB/sec zip pipeline throughput = 131.960'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1016.99 avg-zval = 312.92 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1023.99 avg-zval = 312.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7732 value = 3.2500 dict = 1.00 all = 3.1509 } Zip/UnZip{ index = 0.1476 value = 0.3077 dict = 1.00 all = 0.3174 } ---------------------------- total value len = 9.908904 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585881 Billion total entry num = 0.053393610 Billion write speed all = 0.579903822 MB/sec (with seq num) write speed all = 0.558118791 MB/sec (without seq num) 2024/11/23-14:21:59.713777 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262925: 129431 keys, 42596152 bytes, temperature: kWarm 2024/11/23-14:21:59.713849 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342919713812, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262925, "file_size": 42596152, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874127, "largest_seqno": 4018351019, "table_properties": {"data_size": 40501392, "index_size": 229312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629574, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342917, "oldest_key_time": 0, "file_creation_time": 1732342917, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262925, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:21:59.752950 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262926.sst) /dev/shm/ToplingTemp/Topling-71322-TZw4Af 2024/11/23-14:22:00.451527 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262926.sst): first pass time = 0.70's, 192.144'MB/sec 2024/11/23-14:22:01.204583 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262926.sst): old prealloc_size = 147639500, real_size = 40590848 2024/11/23-14:22:01.204706 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262926.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.83 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.98 avg-zval = 311.53 2024/11/23-14:22:01.204856 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262926.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:22:01.372131 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262926.sst): second pass time = 0.75's, 174.941'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 253.238'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2284.366'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.05's, 891.648'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46446.999'MB/sec zip my value time = 0.75's, unzip length = 0.132'GB zip my value throughput = 174.941'MB/sec zip pipeline throughput = 174.941'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.83 avg-val = 1016.98 avg-zval = 311.53 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.83 avg-val = 1023.98 avg-zval = 311.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129432 bytes = 485392 } seq expand size = 906024 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5733 value = 3.2645 dict = 1.00 all = 3.1638 } Zip/UnZip{ index = 0.1521 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.041440 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715313 Billion total entry num = 0.053523042 Billion write speed all = 0.586744139 MB/sec (with seq num) write speed all = 0.564908202 MB/sec (without seq num) 2024/11/23-14:22:01.424084 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262926: 129432 keys, 42423667 bytes, temperature: kWarm 2024/11/23-14:22:01.424157 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342921424119, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262926, "file_size": 42423667, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873605, "largest_seqno": 4018343835, "table_properties": {"data_size": 40322032, "index_size": 236288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629466, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342919, "oldest_key_time": 0, "file_creation_time": 1732342919, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262926, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:01.461538 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262927.sst) /dev/shm/ToplingTemp/Topling-71322-Tv2zle 2024/11/23-14:22:02.126912 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262927.sst): first pass time = 0.67's, 201.731'MB/sec 2024/11/23-14:22:02.972969 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262927.sst): old prealloc_size = 147639500, real_size = 40722912 2024/11/23-14:22:02.973139 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262927.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.99 avg-zval = 312.57 2024/11/23-14:22:02.973329 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262927.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:22:03.216214 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262927.sst): second pass time = 0.84's, 155.928'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.24's, 174.962'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1748.903'MB/sec (index lex order gen) rebuild zvType time = 0.20's, 0.165'MB/sec write SST data time = 0.05's, 904.799'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44906.356'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 155.928'MB/sec zip pipeline throughput = 155.928'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.99 avg-zval = 312.57 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.99 avg-zval = 312.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.6379 value = 3.2536 dict = 1.00 all = 3.1540 } Zip/UnZip{ index = 0.1507 value = 0.3074 dict = 1.00 all = 0.3171 } ---------------------------- total value len = 10.173975 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844743 Billion total entry num = 0.053652472 Billion write speed all = 0.593579178 MB/sec (with seq num) write speed all = 0.571692494 MB/sec (without seq num) 2024/11/23-14:22:03.268131 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262927: 129430 keys, 42555390 bytes, temperature: kWarm 2024/11/23-14:22:03.268203 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342923268166, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262927, "file_size": 42555390, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4018350359, "table_properties": {"data_size": 40456400, "index_size": 233984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629337, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342921, "oldest_key_time": 0, "file_creation_time": 1732342921, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262927, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:03.304914 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262928.sst) /dev/shm/ToplingTemp/Topling-71322-mejwbf 2024/11/23-14:22:04.030196 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262928.sst): first pass time = 0.73's, 185.066'MB/sec 2024/11/23-14:22:04.858905 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262928.sst): old prealloc_size = 147639500, real_size = 41007232 2024/11/23-14:22:04.859046 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262928.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.00 avg-zval = 314.84 2024/11/23-14:22:04.859208 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262928.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:22:05.019283 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262928.sst): second pass time = 0.83's, 158.983'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.159'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1852.362'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 1008.572'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47284.003'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 158.983'MB/sec zip pipeline throughput = 158.983'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1017.00 avg-zval = 314.84 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1024.00 avg-zval = 314.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.8826 value = 3.2303 dict = 1.00 all = 3.1330 } Zip/UnZip{ index = 0.1453 value = 0.3096 dict = 1.00 all = 0.3192 } ---------------------------- total value len = 10.306511 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974172 Billion total entry num = 0.053781901 Billion write speed all = 0.600414206 MB/sec (with seq num) write speed all = 0.578476741 MB/sec (without seq num) 2024/11/23-14:22:05.071540 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262928: 129429 keys, 42839729 bytes, temperature: kWarm 2024/11/23-14:22:05.071611 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342925071574, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262928, "file_size": 42839729, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237871365, "largest_seqno": 4018338674, "table_properties": {"data_size": 40749040, "index_size": 225664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629753, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342923, "oldest_key_time": 0, "file_creation_time": 1732342923, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262928, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:05.115288 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262929.sst) /dev/shm/ToplingTemp/Topling-71322-Nv3wbg 2024/11/23-14:22:05.825470 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262929.sst): first pass time = 0.71's, 189.001'MB/sec 2024/11/23-14:22:06.692506 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262929.sst): old prealloc_size = 147639500, real_size = 40565248 2024/11/23-14:22:06.692662 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262929.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.78 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.01 avg-zval = 311.39 2024/11/23-14:22:06.692828 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262929.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:22:06.946026 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262929.sst): second pass time = 0.87's, 151.958'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 166.735'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1701.158'MB/sec (index lex order gen) rebuild zvType time = 0.21's, 0.155'MB/sec write SST data time = 0.05's, 937.732'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46611.729'MB/sec zip my value time = 0.87's, unzip length = 0.132'GB zip my value throughput = 151.958'MB/sec zip pipeline throughput = 158.389'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.78 avg-val = 1017.01 avg-zval = 311.39 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.78 avg-val = 1021.81 avg-zval = 311.39 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 88614 bytes = 332320 } seq expand size = 620298 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0423 GB } UnZip/Zip{ index = 6.7354 value = 3.2661 dict = 1.00 all = 3.1756 } Zip/UnZip{ index = 0.1485 value = 0.3062 dict = 1.00 all = 0.3149 } ---------------------------- total value len = 10.438761 GB avg = 0.194 KB (by entry num) total key len = 1.471884 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103600 Billion total entry num = 0.053911329 Billion write speed all = 0.607229589 MB/sec (with seq num) write speed all = 0.585241492 MB/sec (without seq num) 2024/11/23-14:22:06.997265 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262929: 129428 keys, 42265464 bytes, temperature: kWarm 2024/11/23-14:22:06.997338 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342926997300, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262929, "file_size": 42265464, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4018343732, "table_properties": {"data_size": 40302128, "index_size": 230592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629900, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342925, "oldest_key_time": 0, "file_creation_time": 1732342925, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262929, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:07.036184 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262930.sst) /dev/shm/ToplingTemp/Topling-71322-HaInHg 2024/11/23-14:22:07.568729 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262930.sst): first pass time = 0.53's, 252.049'MB/sec 2024/11/23-14:22:08.362007 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262930.sst): old prealloc_size = 147639500, real_size = 40778064 2024/11/23-14:22:08.362126 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262930.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.70 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1016.98 avg-zval = 313.10 2024/11/23-14:22:08.362234 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262930.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:22:08.520310 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262930.sst): second pass time = 0.79's, 166.079'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.845'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2186.038'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 1031.437'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.379'MB, throughput = 46718.575'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 166.079'MB/sec zip pipeline throughput = 166.079'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.70 avg-val = 1016.98 avg-zval = 313.10 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.70 avg-val = 1020.59 avg-zval = 313.10 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 66880 bytes = 250816 } seq expand size = 468160 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 7.0466 value = 3.2481 dict = 1.00 all = 3.1657 } Zip/UnZip{ index = 0.1419 value = 0.3079 dict = 1.00 all = 0.3159 } ---------------------------- total value len = 10.570858 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233032 Billion total entry num = 0.054040761 Billion write speed all = 0.614046875 MB/sec (with seq num) write speed all = 0.592007756 MB/sec (without seq num) 2024/11/23-14:22:08.571455 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262930: 129432 keys, 42397084 bytes, temperature: kWarm 2024/11/23-14:22:08.571524 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342928571488, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262930, "file_size": 42397084, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4018247581, "table_properties": {"data_size": 40525120, "index_size": 220416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629109, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342927, "oldest_key_time": 0, "file_creation_time": 1732342927, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262930, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:08.606426 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262931.sst) /dev/shm/ToplingTemp/Topling-71322-DhRkPd 2024/11/23-14:22:09.239526 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262931.sst): first pass time = 0.63's, 212.014'MB/sec 2024/11/23-14:22:10.033002 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262931.sst): old prealloc_size = 147639500, real_size = 40792752 2024/11/23-14:22:10.033119 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262931.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.02 avg-zval = 313.20 2024/11/23-14:22:10.033235 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262931.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:22:10.210073 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262931.sst): second pass time = 0.79's, 166.246'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 239.826'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2363.523'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.06's, 725.397'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49816.424'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 166.246'MB/sec zip pipeline throughput = 166.246'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.02 avg-zval = 313.20 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1021.32 avg-zval = 313.20 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 79495 bytes = 298128 } seq expand size = 556465 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.9534 value = 3.2472 dict = 1.00 all = 3.1611 } Zip/UnZip{ index = 0.1438 value = 0.3080 dict = 1.00 all = 0.3163 } ---------------------------- total value len = 10.703044 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362458 Billion total entry num = 0.054170187 Billion write speed all = 0.620863916 MB/sec (with seq num) write speed all = 0.598773917 MB/sec (without seq num) 2024/11/23-14:22:10.264697 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262931: 129426 keys, 42458846 bytes, temperature: kWarm 2024/11/23-14:22:10.264789 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342930264739, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262931, "file_size": 42458846, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4018331034, "table_properties": {"data_size": 40536864, "index_size": 223360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629282, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342928, "oldest_key_time": 0, "file_creation_time": 1732342928, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262931, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:10.315366 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262932.sst) /dev/shm/ToplingTemp/Topling-71322-zw0w7e 2024/11/23-14:22:11.012473 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262932.sst): first pass time = 0.70's, 192.555'MB/sec 2024/11/23-14:22:11.845533 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262932.sst): old prealloc_size = 147639500, real_size = 41064928 2024/11/23-14:22:11.845654 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262932.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.03 avg-zval = 315.32 2024/11/23-14:22:11.845807 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262932.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:22:12.009427 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262932.sst): second pass time = 0.83's, 158.146'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.680'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2289.313'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.04's, 982.520'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 50407.118'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 158.146'MB/sec zip pipeline throughput = 158.146'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.03 avg-zval = 315.32 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1023.51 avg-zval = 315.32 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 119823 bytes = 449344 } seq expand size = 838761 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9814 value = 3.2254 dict = 1.00 all = 3.1299 } Zip/UnZip{ index = 0.1432 value = 0.3100 dict = 1.00 all = 0.3195 } ---------------------------- total value len = 10.835512 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021175 GB avg = 0.015 KB total ukey num = 0.067491883 Billion total entry num = 0.054299612 Billion write speed all = 0.627690661 MB/sec (with seq num) write speed all = 0.605549915 MB/sec (without seq num) 2024/11/23-14:22:12.061331 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262932: 129425 keys, 42882116 bytes, temperature: kWarm 2024/11/23-14:22:12.061406 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342932061364, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262932, "file_size": 42882116, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4018332178, "table_properties": {"data_size": 40809936, "index_size": 222464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629252, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342930, "oldest_key_time": 0, "file_creation_time": 1732342930, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262932, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:12.098907 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262933.sst) /dev/shm/ToplingTemp/Topling-71322-G86gDf 2024/11/23-14:22:12.807130 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262933.sst): first pass time = 0.71's, 189.525'MB/sec 2024/11/23-14:22:13.605168 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262933.sst): old prealloc_size = 147639500, real_size = 40546352 2024/11/23-14:22:13.605289 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262933.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.25 2024/11/23-14:22:13.605412 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262933.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:22:13.768027 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262933.sst): second pass time = 0.80's, 165.095'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.981'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2334.725'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 937.171'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45377.866'MB/sec zip my value time = 0.80's, unzip length = 0.132'GB zip my value throughput = 165.095'MB/sec zip pipeline throughput = 165.095'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1017.00 avg-zval = 311.25 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1020.65 avg-zval = 311.25 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 67514 bytes = 253200 } seq expand size = 472598 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 6.8054 value = 3.2674 dict = 1.00 all = 3.1830 } Zip/UnZip{ index = 0.1469 value = 0.3061 dict = 1.00 all = 0.3142 } ---------------------------- total value len = 10.967614 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621313 Billion total entry num = 0.054429042 Billion write speed all = 0.634498846 MB/sec (with seq num) write speed all = 0.612307314 MB/sec (without seq num) 2024/11/23-14:22:13.820061 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262933: 129430 keys, 42167047 bytes, temperature: kWarm 2024/11/23-14:22:13.820135 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342933820095, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262933, "file_size": 42167047, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4018346100, "table_properties": {"data_size": 40285600, "index_size": 228224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629810, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342932, "oldest_key_time": 0, "file_creation_time": 1732342932, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262933, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:13.858998 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262934.sst) /dev/shm/ToplingTemp/Topling-71322-64C5Be 2024/11/23-14:22:14.542019 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262934.sst): first pass time = 0.68's, 196.520'MB/sec 2024/11/23-14:22:15.327285 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262934.sst): old prealloc_size = 147639500, real_size = 40614960 2024/11/23-14:22:15.327433 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262934.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.01 avg-zval = 311.82 2024/11/23-14:22:15.327526 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262934.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:22:15.488852 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262934.sst): second pass time = 0.78's, 167.761'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.791'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1782.404'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 964.639'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47112.911'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 167.761'MB/sec zip pipeline throughput = 167.761'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.01 avg-zval = 311.82 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1019.13 avg-zval = 311.82 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 39203 bytes = 147024 } seq expand size = 274421 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9336 value = 3.2615 dict = 1.00 all = 3.1858 } Zip/UnZip{ index = 0.1442 value = 0.3066 dict = 1.00 all = 0.3139 } ---------------------------- total value len = 11.099519 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750741 Billion total entry num = 0.054558470 Billion write speed all = 0.641296945 MB/sec (with seq num) write speed all = 0.619054594 MB/sec (without seq num) 2024/11/23-14:22:15.582808 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262934: 129428 keys, 42129873 bytes, temperature: kWarm 2024/11/23-14:22:15.582857 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342935582826, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262934, "file_size": 42129873, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4018347741, "table_properties": {"data_size": 40358432, "index_size": 224000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629851, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342933, "oldest_key_time": 0, "file_creation_time": 1732342933, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262934, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:15.623740 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262935.sst) /dev/shm/ToplingTemp/Topling-71322-vtWh3e 2024/11/23-14:22:16.204308 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262935.sst): first pass time = 0.58's, 231.197'MB/sec 2024/11/23-14:22:16.995422 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262935.sst): old prealloc_size = 147639500, real_size = 40465728 2024/11/23-14:22:16.995557 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262935.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.67 2024/11/23-14:22:16.995645 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262935.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:22:17.155084 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262935.sst): second pass time = 0.79's, 166.540'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.774'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2018.553'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.04's, 974.416'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 51086.172'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 166.540'MB/sec zip pipeline throughput = 166.540'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 310.67 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.70 avg-zval = 310.67 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 31758 bytes = 119104 } seq expand size = 222306 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9636 value = 3.2735 dict = 1.00 all = 3.1993 } Zip/UnZip{ index = 0.1436 value = 0.3055 dict = 1.00 all = 0.3126 } ---------------------------- total value len = 11.231371 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880172 Billion total entry num = 0.054687901 Billion write speed all = 0.648092992 MB/sec (with seq num) write speed all = 0.625799768 MB/sec (without seq num) 2024/11/23-14:22:17.207093 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262935: 129431 keys, 41952883 bytes, temperature: kWarm 2024/11/23-14:22:17.207157 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342937207124, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262935, "file_size": 41952883, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4018349396, "table_properties": {"data_size": 40210160, "index_size": 223040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629644, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342935, "oldest_key_time": 0, "file_creation_time": 1732342935, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262935, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:17.247198 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262936.sst) /dev/shm/ToplingTemp/Topling-71322-QkPjvf 2024/11/23-14:22:17.943862 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262936.sst): first pass time = 0.70's, 192.670'MB/sec 2024/11/23-14:22:18.844423 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262936.sst): old prealloc_size = 147639500, real_size = 40451056 2024/11/23-14:22:18.844545 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262936.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.01 avg-zval = 310.57 2024/11/23-14:22:18.844623 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262936.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:22:19.002338 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262936.sst): second pass time = 0.90's, 146.282'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.416'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2208.631'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.280'MB/sec write SST data time = 0.04's, 996.292'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 51858.540'MB/sec zip my value time = 0.90's, unzip length = 0.132'GB zip my value throughput = 146.282'MB/sec zip pipeline throughput = 146.282'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.01 avg-zval = 310.57 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.34 avg-zval = 310.57 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 24684 bytes = 92576 } seq expand size = 172788 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.9957 value = 3.2747 dict = 1.00 all = 3.2024 } Zip/UnZip{ index = 0.1429 value = 0.3054 dict = 1.00 all = 0.3123 } ---------------------------- total value len = 11.363174 GB avg = 0.207 KB (by entry num) total key len = 1.490004 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009601 Billion total entry num = 0.054817330 Billion write speed all = 0.654879348 MB/sec (with seq num) write speed all = 0.632535465 MB/sec (without seq num) 2024/11/23-14:22:19.074745 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262936: 129429 keys, 41911775 bytes, temperature: kWarm 2024/11/23-14:22:19.074815 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342939074777, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262936, "file_size": 41911775, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4018302762, "table_properties": {"data_size": 40196512, "index_size": 222016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630177, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342937, "oldest_key_time": 0, "file_creation_time": 1732342937, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262936, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:19.113427 139957475337792 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262937.sst) /dev/shm/ToplingTemp/Topling-71322-hzRYbf 2024/11/23-14:22:19.133600 139957475337792 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262937.sst): first pass time = 0.02's, 514.396'MB/sec 2024/11/23-14:22:19.242069 139957475337792 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262937.sst): old prealloc_size = 147639500, real_size = 3209984 2024/11/23-14:22:19.242125 139957475337792 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262937.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 9988 prefix = 4 raw-key = 0.0001 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 1.85 raw-val = 0.0102 GB zip-val = 0.0032 GB avg-val =1017.09 avg-zval = 319.27 2024/11/23-14:22:19.242169 139957475337792 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262937.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0000 GB, waited 0.000 sec, Key+Value = 0.010 GB 2024/11/23-14:22:19.256338 139957475337792 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00/262937.sst): second pass time = 0.11's, 94.187'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.01's, 233.449'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 519.124'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.233'MB/sec write SST data time = 0.00's, 953.115'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.113'MB, throughput = 4706.022'MB/sec zip my value time = 0.11's, unzip length = 0.010'GB zip my value throughput = 94.187'MB/sec zip pipeline throughput = 94.187'MB/sec entries = 9988 avg-key = 20.00 avg-zkey = 1.85 avg-val = 1017.09 avg-zval = 319.27 usrkeys = 9988 avg-key = 12.00 avg-zkey = 1.85 avg-val = 1018.38 avg-zval = 319.27 TagRS{ kinds = RS_Key0_TagN bytes = 1680 } TagArray{ size = 1843 bytes = 6928 } seq expand size = 12901 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0001 GB value = 0.0102 GB dict = 0.11 MB all = 0.0104 GB } __ZipSize{ index = 0.0000 GB value = 0.0032 GB dict = 0.11 MB all = 0.0033 GB } UnZip/Zip{ index = 6.5026 value = 3.1857 dict = 1.00 all = 3.1105 } Zip/UnZip{ index = 0.1538 value = 0.3139 dict = 1.00 all = 0.3215 } ---------------------------- total value len = 11.373345 GB avg = 0.207 KB (by entry num) total key len = 1.490204 GB avg = 0.027 KB total ukey len = 1.027508 GB avg = 0.015 KB total ukey num = 0.068019589 Billion total entry num = 0.054827318 Billion write speed all = 0.655399295 MB/sec (with seq num) write speed all = 0.633051631 MB/sec (without seq num) 2024/11/23-14:22:19.267034 139957475337792 [compaction_job.cc:1948] [default] [JOB 27621] Generated table #262937: 9988 keys, 3330197 bytes, temperature: kWarm 2024/11/23-14:22:19.267082 139957475337792 EVENT_LOG_v1 {"time_micros": 1732342939267054, "cf_name": "default", "job": 27621, "event": "table_file_creation", "file_number": 262937, "file_size": 3330197, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4018350273, "table_properties": {"data_size": 3188848, "index_size": 18432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 199760, "raw_average_key_size": 20, "raw_value_size": 10158723, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 9988, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732342939, "oldest_key_time": 0, "file_creation_time": 1732342939, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 262937, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:22:19.278377 139957475337792 [compaction_job.cc:703] job-27621: subcompact[0], size: 0.810321 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-27621/att-00 [262918,262919,262920,262921,262922,262923,262924,262925,262926,262927,262928,262929,262930,262931,262932,262933,262934,262935,262936,262937] 2024/11/23-14:22:19.435957 139957475337792 (Original Log Time 2024/11/23-14:21:42.697848) EVENT_LOG_v1 {"time_micros": 1732342902697819, "job": 27621, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [262912], "files_L3": [262865, 262866, 262867, 262868, 262869, 262870, 262871, 262872, 262873, 262874, 262875, 262876, 262877, 262878, 262879, 262880, 262881, 262882, 262883], "score": 0.997765, "input_data_size": 839683642, "oldest_snapshot_seqno": 4018400065} 2024/11/23-14:22:19.435961 139957475337792 (Original Log Time 2024/11/23-14:22:19.430780) [compaction_job.cc:2022] [default] [JOB 27621] Compacted 1@2 + 19@3 files to L3 => 810320973 bytes 2024/11/23-14:22:19.435962 139957475337792 (Original Log Time 2024/11/23-14:22:19.435885) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 20 0 0 0] max score 0.03, MB/sec: 23.0 rd, 22.2 wr, level 3, files in(1, 19) out(20 +0 blob) MB in(31.2, 769.6 +0.0 blob) out(772.8 +0.0 blob), read-write-amplify(50.5) write-amplify(24.8) OK, records in: 2491074, records dropped: 21920 output_compression: Snappy 2024/11/23-14:22:19.435966 139957475337792 (Original Log Time 2024/11/23-14:22:19.435930) EVENT_LOG_v1 {"time_micros": 1732342939435909, "cf": "default", "job": 27621, "event": "compaction_finished", "compaction_time_micros": 36580533, "compaction_time_cpu_micros": 7602538, "output_level": 3, "num_output_files": 20, "total_output_size": 810320973, "num_input_records": 2491074, "num_output_records": 2469154, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 648071037, "file_range_sync_nanos": 0, "file_fsync_nanos": 14380, "file_prepare_write_nanos": 47979, "lsm_state": [0, 0, 0, 20, 0, 0, 0]} 2024/11/23-14:22:19.554287 139957475337792 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732342902,"executesMs":36875,"compactionJobId":27621,"attempt":0,"compactionInputRawBytes":2583234196,"compactionInputZipBytes":839683642,"compactionOutputRawBytes":0,"compactionOutputZipBytes":810320973,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-687c9319a0fd31f36e188e2e3d744cee-aebccf91bc0f22e1-00"}}, response = 2024/11/23-14:22:19.554374 139957475337792 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-27621/att-00: olev 3, work 36.875 s, result 7.585 ms, install 5.202 ms, input{raw 2.406 GiB zip 800.785 MiB}