2024/11/24-08:18:34.420015 139957492123200 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00: fork to child time = 0.006300 sec 2024/11/24-08:18:34.420630 139957492123200 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00 2024/11/24-08:18:34.426722 139957492123200 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00 2024/11/24-08:18:34.427578 139957492123200 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/24-08:18:34.427776 139957492123200 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/24-08:18:34.481430 139957492123200 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-43894/att-00/MANIFEST-206347 2024/11/24-08:18:34.499907 139957492123200 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-43894/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 297408, last_sequence is 4511594905, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 297233 2024/11/24-08:18:34.499913 139957492123200 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/24-08:18:34.500061 139957492123200 [version_set.cc:5721] Creating manifest 297408 2024/11/24-08:18:34.531315 139957492123200 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/24-08:18:34.552768 139957492123200 [compaction_job.cc:2437] [default] [JOB 43894] Compacting 1@2 + 29@3 files to L3, score 0.80, subcompactions 1 : 1 2024/11/24-08:18:34.552799 139957492123200 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 32905 Base level 2, inputs: [297372(27MB)], [297386(36MB) 297387(3212KB) 297314(35MB) 297315(33MB) 297316(35MB) 297317(33MB) 297318(35MB) 297319(33MB) 297320(35MB) 297321(30MB) 297322(30MB) 297323(35MB) 297324(33MB) 297325(30MB) 297326(35MB) 297327(33MB) 297328(32MB) 297329(35MB) 297330(33MB) 297331(32MB) 297332(33MB) 297333(33MB) 297334(34MB) 297335(33MB) 297336(33MB) 297337(14MB) 297298(34MB) 297299(33MB) 297397(34MB)] 2024/11/24-08:18:34.561838 139957492123200 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-08:18:34.561878 139957492123200 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-08:18:34.561902 139957492123200 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/24-08:18:34.561910 139957492123200 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-08:18:34.561924 139957492123200 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/24-08:18:35.144009 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297409.sst) /dev/shm/ToplingTemp/Topling-304735-6dPPSQ 2024/11/24-08:18:35.690109 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297409.sst): first pass time = 0.55's, 245.794'MB/sec 2024/11/24-08:18:37.343685 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297409.sst): old prealloc_size = 147639500, real_size = 34543856 2024/11/24-08:18:37.344270 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297409.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924331 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0325 GB avg-val = 117.21 avg-zval = 35.21 2024/11/24-08:18:37.345078 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297409.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:18:38.523020 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297409.sst): second pass time = 1.65's, 65.548'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.18's, 32.663'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3129.463'MB/sec (index lex order gen) rebuild zvType time = 0.98's, 0.235'MB/sec write SST data time = 0.20's, 197.312'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 34994.500'MB/sec zip my value time = 1.65's, unzip length = 0.108'GB zip my value throughput = 65.548'MB/sec zip pipeline throughput = 65.548'MB/sec entries = 924331 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.21 avg-zval = 35.21 usrkeys = 924331 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.21 avg-zval = 35.21 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924331 bytes = 3119632 } seq expand size = 6470317 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0325 GB dict = 1.08 MB all = 0.0385 GB } UnZip/Zip{ index = 10.4438 value = 3.3291 dict = 1.00 all = 3.4844 } Zip/UnZip{ index = 0.0958 value = 0.3004 dict = 1.00 all = 0.2870 } ---------------------------- total value len = 2251.440380 GB avg = 0.200 KB (by entry num) total key len = 306.908844 GB avg = 0.027 KB total ukey len = 237.373214 GB avg = 0.015 KB total ukey num = 15.739585494 Billion total entry num = 11.280899442 Billion write speed all = 30.381918811 MB/sec (with seq num) write speed all = 29.310179682 MB/sec (without seq num) 2024/11/24-08:18:38.572582 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297409: 924331 keys, 38519904 bytes, temperature: kWarm 2024/11/24-08:18:38.572662 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407518572611, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297409, "file_size": 38519904, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4408257017, "largest_seqno": 4511409840, "table_properties": {"data_size": 32542496, "index_size": 1770112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25881268, "raw_average_key_size": 28, "raw_value_size": 108336594, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924331, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407515, "oldest_key_time": 0, "file_creation_time": 1732407515, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297409, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:18:38.610864 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297410.sst) /dev/shm/ToplingTemp/Topling-304735-1O2CqR 2024/11/24-08:18:39.651700 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297410.sst): first pass time = 1.04's, 128.956'MB/sec 2024/11/24-08:18:41.061541 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297410.sst): old prealloc_size = 147639500, real_size = 33107184 2024/11/24-08:18:41.062133 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297410.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927108 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.77 avg-zval = 33.56 2024/11/24-08:18:41.062885 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297410.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:18:42.046806 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297410.sst): second pass time = 1.41's, 76.827'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 37.764'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3117.881'MB/sec (index lex order gen) rebuild zvType time = 0.85's, 0.272'MB/sec write SST data time = 0.13's, 283.586'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34945.751'MB/sec zip my value time = 1.41's, unzip length = 0.108'GB zip my value throughput = 76.827'MB/sec zip pipeline throughput = 76.827'MB/sec entries = 927108 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.77 avg-zval = 33.56 usrkeys = 927108 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.77 avg-zval = 33.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927108 bytes = 3244896 } seq expand size = 6489756 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5135 value = 3.4797 dict = 1.00 all = 3.6074 } Zip/UnZip{ index = 0.0951 value = 0.2874 dict = 1.00 all = 0.2772 } ---------------------------- total value len = 2251.555129 GB avg = 0.200 KB (by entry num) total key len = 306.934803 GB avg = 0.027 KB total ukey len = 237.391756 GB avg = 0.015 KB total ukey num = 15.740512602 Billion total entry num = 11.281826550 Billion write speed all = 30.382318384 MB/sec (with seq num) write speed all = 29.310536027 MB/sec (without seq num) 2024/11/24-08:18:42.095496 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297410: 927108 keys, 37206438 bytes, temperature: kWarm 2024/11/24-08:18:42.095566 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407522095529, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297410, "file_size": 37206438, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680186, "largest_seqno": 4511409940, "table_properties": {"data_size": 31111584, "index_size": 1763648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25959024, "raw_average_key_size": 28, "raw_value_size": 108258844, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927108, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407518, "oldest_key_time": 0, "file_creation_time": 1732407518, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297410, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:18:42.132453 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297411.sst) /dev/shm/ToplingTemp/Topling-304735-6l1EfO 2024/11/24-08:18:43.071659 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297411.sst): first pass time = 0.94's, 142.911'MB/sec 2024/11/24-08:18:44.404955 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297411.sst): old prealloc_size = 147639500, real_size = 31365504 2024/11/24-08:18:44.405529 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297411.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921171 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0294 GB avg-val = 117.70 avg-zval = 31.88 2024/11/24-08:18:44.406296 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297411.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:18:45.337044 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297411.sst): second pass time = 1.33's, 81.371'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 38.031'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3296.650'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.287'MB/sec write SST data time = 0.13's, 276.386'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 33906.392'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.371'MB/sec zip pipeline throughput = 81.371'MB/sec entries = 921171 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.70 avg-zval = 31.88 usrkeys = 921171 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.70 avg-zval = 31.88 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921171 bytes = 3224112 } seq expand size = 6448197 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0354 GB } UnZip/Zip{ index = 10.4470 value = 3.6915 dict = 1.00 all = 3.7864 } Zip/UnZip{ index = 0.0957 value = 0.2709 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 2251.670002 GB avg = 0.200 KB (by entry num) total key len = 306.960596 GB avg = 0.027 KB total ukey len = 237.410180 GB avg = 0.015 KB total ukey num = 15.741433773 Billion total entry num = 11.282747721 Billion write speed all = 30.382801699 MB/sec (with seq num) write speed all = 29.310973708 MB/sec (without seq num) 2024/11/24-08:18:45.382090 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297411: 921171 keys, 35447030 bytes, temperature: kWarm 2024/11/24-08:18:45.382158 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407525382124, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297411, "file_size": 35447030, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4373312907, "largest_seqno": 4511409116, "table_properties": {"data_size": 29371504, "index_size": 1763520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25792788, "raw_average_key_size": 28, "raw_value_size": 108424986, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921171, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407522, "oldest_key_time": 0, "file_creation_time": 1732407522, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297411, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:18:45.419238 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297412.sst) /dev/shm/ToplingTemp/Topling-304735-gQWJ3O 2024/11/24-08:18:46.365779 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297412.sst): first pass time = 0.95's, 141.803'MB/sec 2024/11/24-08:18:47.753072 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297412.sst): old prealloc_size = 147639500, real_size = 33087088 2024/11/24-08:18:47.753651 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297412.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926640 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.84 avg-zval = 33.54 2024/11/24-08:18:47.754438 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297412.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:18:48.602247 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297412.sst): second pass time = 1.39's, 78.094'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.791'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3193.339'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 291.687'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34441.717'MB/sec zip my value time = 1.39's, unzip length = 0.108'GB zip my value throughput = 78.094'MB/sec zip pipeline throughput = 78.094'MB/sec entries = 926640 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.84 avg-zval = 33.54 usrkeys = 926640 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.84 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926640 bytes = 3243248 } seq expand size = 6486480 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4419 value = 3.4836 dict = 1.00 all = 3.6095 } Zip/UnZip{ index = 0.0958 value = 0.2871 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 2251.784760 GB avg = 0.200 KB (by entry num) total key len = 306.986541 GB avg = 0.027 KB total ukey len = 237.428712 GB avg = 0.015 KB total ukey num = 15.742360413 Billion total entry num = 11.283674361 Billion write speed all = 30.383294451 MB/sec (with seq num) write speed all = 29.311419992 MB/sec (without seq num) 2024/11/24-08:18:48.650462 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297412: 926640 keys, 37184822 bytes, temperature: kWarm 2024/11/24-08:18:48.650533 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407528650497, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297412, "file_size": 37184822, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680199, "largest_seqno": 4511406719, "table_properties": {"data_size": 31080416, "index_size": 1774848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25945920, "raw_average_key_size": 28, "raw_value_size": 108271844, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926640, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407525, "oldest_key_time": 0, "file_creation_time": 1732407525, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297412, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:18:48.688803 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297413.sst) /dev/shm/ToplingTemp/Topling-304735-VV3ifP 2024/11/24-08:18:49.675548 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297413.sst): first pass time = 0.99's, 136.026'MB/sec 2024/11/24-08:18:50.983073 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297413.sst): old prealloc_size = 147639500, real_size = 31190608 2024/11/24-08:18:50.983585 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297413.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921675 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.62 avg-zval = 31.68 2024/11/24-08:18:50.984349 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297413.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:18:51.792123 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297413.sst): second pass time = 1.31's, 82.970'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.601'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3597.115'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.339'MB/sec write SST data time = 0.13's, 275.525'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32189.444'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.970'MB/sec zip pipeline throughput = 82.970'MB/sec entries = 921675 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.62 avg-zval = 31.68 usrkeys = 921675 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.62 avg-zval = 31.68 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921675 bytes = 3225872 } seq expand size = 6451725 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4481 value = 3.7132 dict = 1.00 all = 3.8050 } Zip/UnZip{ index = 0.0957 value = 0.2693 dict = 1.00 all = 0.2628 } ---------------------------- total value len = 2251.899623 GB avg = 0.200 KB (by entry num) total key len = 307.012348 GB avg = 0.027 KB total ukey len = 237.447146 GB avg = 0.015 KB total ukey num = 15.743282088 Billion total entry num = 11.284596036 Billion write speed all = 30.383813933 MB/sec (with seq num) write speed all = 29.311892522 MB/sec (without seq num) 2024/11/24-08:18:51.835588 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297413: 921675 keys, 35273754 bytes, temperature: kWarm 2024/11/24-08:18:51.835660 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407531835623, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297413, "file_size": 35273754, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4376383524, "largest_seqno": 4511409131, "table_properties": {"data_size": 29195712, "index_size": 1764288, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25806900, "raw_average_key_size": 28, "raw_value_size": 108410942, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921675, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407528, "oldest_key_time": 0, "file_creation_time": 1732407528, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297413, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:18:51.872288 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297414.sst) /dev/shm/ToplingTemp/Topling-304735-xtQIMR 2024/11/24-08:18:52.878550 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297414.sst): first pass time = 1.01's, 133.388'MB/sec 2024/11/24-08:18:54.215392 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297414.sst): old prealloc_size = 147639500, real_size = 33036688 2024/11/24-08:18:54.215962 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297414.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925833 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.97 avg-zval = 33.53 2024/11/24-08:18:54.216730 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297414.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:18:55.062304 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297414.sst): second pass time = 1.34's, 81.076'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.845'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3115.776'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 293.689'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31827.004'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 81.076'MB/sec zip pipeline throughput = 81.076'MB/sec entries = 925833 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.97 avg-zval = 33.53 usrkeys = 925833 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.97 avg-zval = 33.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925833 bytes = 3240432 } seq expand size = 6480831 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5139 value = 3.4884 dict = 1.00 all = 3.6146 } Zip/UnZip{ index = 0.0951 value = 0.2867 dict = 1.00 all = 0.2767 } ---------------------------- total value len = 2252.014398 GB avg = 0.200 KB (by entry num) total key len = 307.038272 GB avg = 0.027 KB total ukey len = 237.465662 GB avg = 0.015 KB total ukey num = 15.744207921 Billion total entry num = 11.285521869 Billion write speed all = 30.384304750 MB/sec (with seq num) write speed all = 29.312337018 MB/sec (without seq num) 2024/11/24-08:18:55.109370 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297414: 925833 keys, 37131776 bytes, temperature: kWarm 2024/11/24-08:18:55.109439 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407535109404, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297414, "file_size": 37131776, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680212, "largest_seqno": 4511409712, "table_properties": {"data_size": 31043904, "index_size": 1761152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25923324, "raw_average_key_size": 28, "raw_value_size": 108294542, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925833, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407531, "oldest_key_time": 0, "file_creation_time": 1732407531, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297414, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:18:55.146033 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297415.sst) /dev/shm/ToplingTemp/Topling-304735-NqeUHO 2024/11/24-08:18:56.058276 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297415.sst): first pass time = 0.91's, 147.135'MB/sec 2024/11/24-08:18:57.366191 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297415.sst): old prealloc_size = 147639500, real_size = 31228608 2024/11/24-08:18:57.366764 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297415.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922484 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.50 avg-zval = 31.69 2024/11/24-08:18:57.367529 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297415.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:18:58.183241 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297415.sst): second pass time = 1.31's, 82.917'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.081'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3206.081'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.334'MB/sec write SST data time = 0.13's, 280.537'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35972.904'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.917'MB/sec zip pipeline throughput = 82.917'MB/sec entries = 922484 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.50 avg-zval = 31.69 usrkeys = 922484 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.50 avg-zval = 31.69 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 922484 bytes = 3113392 } seq expand size = 6457388 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4444 value = 3.7079 dict = 1.00 all = 3.8132 } Zip/UnZip{ index = 0.0957 value = 0.2697 dict = 1.00 all = 0.2623 } ---------------------------- total value len = 2252.129244 GB avg = 0.200 KB (by entry num) total key len = 307.064101 GB avg = 0.027 KB total ukey len = 237.484112 GB avg = 0.015 KB total ukey num = 15.745130405 Billion total entry num = 11.286444353 Billion write speed all = 30.384849093 MB/sec (with seq num) write speed all = 29.312833462 MB/sec (without seq num) 2024/11/24-08:18:58.227445 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297415: 922484 keys, 35198668 bytes, temperature: kWarm 2024/11/24-08:18:58.227515 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407538227479, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297415, "file_size": 35198668, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4380687689, "largest_seqno": 4511409143, "table_properties": {"data_size": 29231344, "index_size": 1766464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25829552, "raw_average_key_size": 28, "raw_value_size": 108388296, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922484, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407535, "oldest_key_time": 0, "file_creation_time": 1732407535, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297415, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:18:58.266578 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297416.sst) /dev/shm/ToplingTemp/Topling-304735-iERx8P 2024/11/24-08:18:59.134486 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297416.sst): first pass time = 0.87's, 154.651'MB/sec 2024/11/24-08:19:00.485897 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297416.sst): old prealloc_size = 147639500, real_size = 33135776 2024/11/24-08:19:00.486436 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297416.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925425 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 117.03 avg-zval = 33.64 2024/11/24-08:19:00.488209 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297416.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:01.326868 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297416.sst): second pass time = 1.35's, 80.188'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 44.276'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3432.260'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.323'MB/sec write SST data time = 0.12's, 298.147'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32269.259'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.188'MB/sec zip pipeline throughput = 80.188'MB/sec entries = 925425 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.03 avg-zval = 33.64 usrkeys = 925425 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.03 avg-zval = 33.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925425 bytes = 3239008 } seq expand size = 6477975 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.4455 value = 3.4789 dict = 1.00 all = 3.6048 } Zip/UnZip{ index = 0.0957 value = 0.2874 dict = 1.00 all = 0.2774 } ---------------------------- total value len = 2252.244028 GB avg = 0.200 KB (by entry num) total key len = 307.090013 GB avg = 0.027 KB total ukey len = 237.502621 GB avg = 0.015 KB total ukey num = 15.746055830 Billion total entry num = 11.287369778 Billion write speed all = 30.385385457 MB/sec (with seq num) write speed all = 29.313321941 MB/sec (without seq num) 2024/11/24-08:19:01.372987 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297416: 925425 keys, 37233190 bytes, temperature: kWarm 2024/11/24-08:19:01.373060 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407541373023, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297416, "file_size": 37233190, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679834, "largest_seqno": 4511406158, "table_properties": {"data_size": 31132304, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25911900, "raw_average_key_size": 28, "raw_value_size": 108305918, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407538, "oldest_key_time": 0, "file_creation_time": 1732407538, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297416, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:01.410228 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297417.sst) /dev/shm/ToplingTemp/Topling-304735-qQDEAO 2024/11/24-08:19:02.357757 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297417.sst): first pass time = 0.95's, 141.656'MB/sec 2024/11/24-08:19:03.648629 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297417.sst): old prealloc_size = 147639500, real_size = 28729616 2024/11/24-08:19:03.649224 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297417.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 940509 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1079 GB zip-val = 0.0267 GB avg-val = 114.71 avg-zval = 28.38 2024/11/24-08:19:03.650026 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297417.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:04.477545 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297417.sst): second pass time = 1.29's, 83.662'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 39.652'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3138.131'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.335'MB/sec write SST data time = 0.13's, 261.234'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 33996.905'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 83.662'MB/sec zip pipeline throughput = 83.662'MB/sec entries = 940509 avg-key = 28.00 avg-zkey = 1.92 avg-val = 114.71 avg-zval = 28.38 usrkeys = 940509 avg-key = 20.00 avg-zkey = 1.92 avg-val = 121.71 avg-zval = 28.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 940509 bytes = 3291792 } seq expand size = 6583563 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0267 GB dict = 1.08 MB all = 0.0329 GB } UnZip/Zip{ index = 10.4434 value = 4.0416 dict = 1.00 all = 4.0837 } Zip/UnZip{ index = 0.0958 value = 0.2474 dict = 1.00 all = 0.2449 } ---------------------------- total value len = 2252.358495 GB avg = 0.200 KB (by entry num) total key len = 307.116347 GB avg = 0.027 KB total ukey len = 237.521431 GB avg = 0.015 KB total ukey num = 15.746996339 Billion total entry num = 11.288310287 Billion write speed all = 30.385920484 MB/sec (with seq num) write speed all = 29.313807742 MB/sec (without seq num) 2024/11/24-08:19:04.520283 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297417: 940509 keys, 32866864 bytes, temperature: kWarm 2024/11/24-08:19:04.520353 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407544520316, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297417, "file_size": 32866864, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265680463, "largest_seqno": 4511407951, "table_properties": {"data_size": 26693184, "index_size": 1801152, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26334252, "raw_average_key_size": 28, "raw_value_size": 107883582, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 940509, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407541, "oldest_key_time": 0, "file_creation_time": 1732407541, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297417, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:04.558613 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297418.sst) /dev/shm/ToplingTemp/Topling-304735-tMg2IN 2024/11/24-08:19:05.509693 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297418.sst): first pass time = 0.95's, 141.127'MB/sec 2024/11/24-08:19:06.768697 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297418.sst): old prealloc_size = 147639500, real_size = 27764080 2024/11/24-08:19:06.769262 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297418.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 945196 prefix = 4 raw-key = 0.0151 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1078 GB zip-val = 0.0257 GB avg-val = 114.00 avg-zval = 27.20 2024/11/24-08:19:06.770025 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297418.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0024 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:07.607342 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297418.sst): second pass time = 1.26's, 85.662'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.84's, 37.914'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3356.844'MB/sec (index lex order gen) rebuild zvType time = 0.71's, 0.332'MB/sec write SST data time = 0.13's, 251.661'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.073'MB, throughput = 33369.311'MB/sec zip my value time = 1.26's, unzip length = 0.108'GB zip my value throughput = 85.662'MB/sec zip pipeline throughput = 85.662'MB/sec entries = 945196 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.00 avg-zval = 27.20 usrkeys = 945196 avg-key = 20.00 avg-zkey = 1.93 avg-val = 121.00 avg-zval = 27.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 945196 bytes = 3190048 } seq expand size = 6616372 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0189 GB value = 0.1078 GB dict = 1.07 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0257 GB dict = 1.07 MB all = 0.0318 GB } UnZip/Zip{ index = 10.3778 value = 4.1917 dict = 1.00 all = 4.2213 } Zip/UnZip{ index = 0.0964 value = 0.2386 dict = 1.00 all = 0.2369 } ---------------------------- total value len = 2252.472864 GB avg = 0.200 KB (by entry num) total key len = 307.142813 GB avg = 0.027 KB total ukey len = 237.540335 GB avg = 0.015 KB total ukey num = 15.747941535 Billion total entry num = 11.289255483 Billion write speed all = 30.386463398 MB/sec (with seq num) write speed all = 29.314300724 MB/sec (without seq num) 2024/11/24-08:19:07.646993 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297418: 945196 keys, 31795376 bytes, temperature: kWarm 2024/11/24-08:19:07.647063 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407547647027, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297418, "file_size": 31795376, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4347310478, "largest_seqno": 4450289312, "table_properties": {"data_size": 25706016, "index_size": 1821568, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26465488, "raw_average_key_size": 28, "raw_value_size": 107752344, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 945196, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 114, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407544, "oldest_key_time": 0, "file_creation_time": 1732407544, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297418, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:07.689754 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297419.sst) /dev/shm/ToplingTemp/Topling-304735-GJa8gP 2024/11/24-08:19:08.701473 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297419.sst): first pass time = 1.01's, 132.667'MB/sec 2024/11/24-08:19:10.035510 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297419.sst): old prealloc_size = 147639500, real_size = 32352080 2024/11/24-08:19:10.036076 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297419.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934253 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0303 GB avg-val = 115.66 avg-zval = 32.48 2024/11/24-08:19:10.036821 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297419.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:10.896463 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297419.sst): second pass time = 1.33's, 81.076'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.364'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3274.433'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.319'MB/sec write SST data time = 0.13's, 283.409'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34364.697'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.076'MB/sec zip pipeline throughput = 81.076'MB/sec entries = 934253 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.66 avg-zval = 32.48 usrkeys = 934253 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.66 avg-zval = 32.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934253 bytes = 3269904 } seq expand size = 6539771 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0365 GB } UnZip/Zip{ index = 10.5118 value = 3.5615 dict = 1.00 all = 3.6800 } Zip/UnZip{ index = 0.0951 value = 0.2808 dict = 1.00 all = 0.2717 } ---------------------------- total value len = 2252.587462 GB avg = 0.200 KB (by entry num) total key len = 307.168972 GB avg = 0.027 KB total ukey len = 237.559020 GB avg = 0.015 KB total ukey num = 15.748875788 Billion total entry num = 11.290189736 Billion write speed all = 30.386947886 MB/sec (with seq num) write speed all = 29.314738351 MB/sec (without seq num) 2024/11/24-08:19:10.941537 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297419: 934253 keys, 36472478 bytes, temperature: kWarm 2024/11/24-08:19:10.941612 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407550941572, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297419, "file_size": 36472478, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679848, "largest_seqno": 4511409532, "table_properties": {"data_size": 30340800, "index_size": 1777536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26159084, "raw_average_key_size": 28, "raw_value_size": 108058666, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934253, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407547, "oldest_key_time": 0, "file_creation_time": 1732407547, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297419, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:10.979345 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297420.sst) /dev/shm/ToplingTemp/Topling-304735-0UIVzP 2024/11/24-08:19:11.787315 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297420.sst): first pass time = 0.81's, 166.124'MB/sec 2024/11/24-08:19:13.073309 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297420.sst): old prealloc_size = 147639500, real_size = 31394784 2024/11/24-08:19:13.073874 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297420.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.97 2024/11/24-08:19:13.074640 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297420.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:13.884910 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297420.sst): second pass time = 1.29's, 84.403'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.707'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3257.464'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.336'MB/sec write SST data time = 0.13's, 280.364'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33585.402'MB/sec zip my value time = 1.29's, unzip length = 0.108'GB zip my value throughput = 84.403'MB/sec zip pipeline throughput = 84.403'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.97 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.97 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3217568 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0355 GB } UnZip/Zip{ index = 10.3764 value = 3.6906 dict = 1.00 all = 3.7838 } Zip/UnZip{ index = 0.0964 value = 0.2710 dict = 1.00 all = 0.2643 } ---------------------------- total value len = 2252.702375 GB avg = 0.200 KB (by entry num) total key len = 307.194712 GB avg = 0.027 KB total ukey len = 237.577406 GB avg = 0.015 KB total ukey num = 15.749795088 Billion total entry num = 11.291109036 Billion write speed all = 30.387539559 MB/sec (with seq num) write speed all = 29.315280758 MB/sec (without seq num) 2024/11/24-08:19:13.929929 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297420: 919300 keys, 35471474 bytes, temperature: kWarm 2024/11/24-08:19:13.930009 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407553929961, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297420, "file_size": 35471474, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4324006899, "largest_seqno": 4473939146, "table_properties": {"data_size": 29392848, "index_size": 1771904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407550, "oldest_key_time": 0, "file_creation_time": 1732407550, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297420, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:13.966318 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297421.sst) /dev/shm/ToplingTemp/Topling-304735-fTYl3O 2024/11/24-08:19:14.837625 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297421.sst): first pass time = 0.87's, 154.048'MB/sec 2024/11/24-08:19:16.121362 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297421.sst): old prealloc_size = 147639500, real_size = 28698224 2024/11/24-08:19:16.121899 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297421.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 939620 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1079 GB zip-val = 0.0267 GB avg-val = 114.84 avg-zval = 28.38 2024/11/24-08:19:16.122694 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297421.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:16.944281 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297421.sst): second pass time = 1.28's, 84.154'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 39.900'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3482.954'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.338'MB/sec write SST data time = 0.13's, 258.955'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.077'MB, throughput = 35254.010'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.154'MB/sec zip pipeline throughput = 84.154'MB/sec entries = 939620 avg-key = 28.00 avg-zkey = 1.91 avg-val = 114.84 avg-zval = 28.38 usrkeys = 939620 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.84 avg-zval = 28.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 939620 bytes = 3288688 } seq expand size = 6577340 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0188 GB value = 0.1079 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0267 GB dict = 1.08 MB all = 0.0328 GB } UnZip/Zip{ index = 10.4439 value = 4.0470 dict = 1.00 all = 4.0879 } Zip/UnZip{ index = 0.0957 value = 0.2471 dict = 1.00 all = 0.2446 } ---------------------------- total value len = 2252.816860 GB avg = 0.200 KB (by entry num) total key len = 307.221022 GB avg = 0.027 KB total ukey len = 237.596198 GB avg = 0.015 KB total ukey num = 15.750734708 Billion total entry num = 11.292048656 Billion write speed all = 30.388107287 MB/sec (with seq num) write speed all = 29.315798198 MB/sec (without seq num) 2024/11/24-08:19:16.985372 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297421: 939620 keys, 32833058 bytes, temperature: kWarm 2024/11/24-08:19:16.985443 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407556985408, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297421, "file_size": 32833058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4366965968, "largest_seqno": 4511408651, "table_properties": {"data_size": 26663776, "index_size": 1799360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26309360, "raw_average_key_size": 28, "raw_value_size": 107908440, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 939620, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407553, "oldest_key_time": 0, "file_creation_time": 1732407553, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297421, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:17.020122 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297422.sst) /dev/shm/ToplingTemp/Topling-304735-XUycoN 2024/11/24-08:19:18.040788 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297422.sst): first pass time = 1.02's, 131.505'MB/sec 2024/11/24-08:19:19.360312 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297422.sst): old prealloc_size = 147639500, real_size = 32184176 2024/11/24-08:19:19.360841 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297422.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934946 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1080 GB zip-val = 0.0302 GB avg-val = 115.56 avg-zval = 32.27 2024/11/24-08:19:19.361672 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297422.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:20.218336 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297422.sst): second pass time = 1.32's, 81.930'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.316'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3430.474'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.320'MB/sec write SST data time = 0.13's, 283.890'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 32861.489'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.930'MB/sec zip pipeline throughput = 81.930'MB/sec entries = 934946 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.56 avg-zval = 32.27 usrkeys = 934946 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.56 avg-zval = 32.27 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934946 bytes = 3272320 } seq expand size = 6544622 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5147 value = 3.5808 dict = 1.00 all = 3.6968 } Zip/UnZip{ index = 0.0951 value = 0.2793 dict = 1.00 all = 0.2705 } ---------------------------- total value len = 2252.931444 GB avg = 0.199 KB (by entry num) total key len = 307.247200 GB avg = 0.027 KB total ukey len = 237.614897 GB avg = 0.015 KB total ukey num = 15.751669654 Billion total entry num = 11.292983602 Billion write speed all = 30.388597152 MB/sec (with seq num) write speed all = 29.316240955 MB/sec (without seq num) 2024/11/24-08:19:20.264082 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297422: 934946 keys, 36306742 bytes, temperature: kWarm 2024/11/24-08:19:20.264155 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407560264117, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297422, "file_size": 36306742, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679865, "largest_seqno": 4511410386, "table_properties": {"data_size": 30171872, "index_size": 1778368, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26178488, "raw_average_key_size": 28, "raw_value_size": 108039340, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934946, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407557, "oldest_key_time": 0, "file_creation_time": 1732407557, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297422, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:20.310345 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297423.sst) /dev/shm/ToplingTemp/Topling-304735-aNDl4P 2024/11/24-08:19:21.131120 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297423.sst): first pass time = 0.82's, 163.534'MB/sec 2024/11/24-08:19:22.410510 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297423.sst): old prealloc_size = 147639500, real_size = 31317392 2024/11/24-08:19:22.411174 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297423.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.90 2024/11/24-08:19:22.412188 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297423.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:23.220945 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297423.sst): second pass time = 1.28's, 84.845'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.81's, 43.533'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2739.722'MB/sec (index lex order gen) rebuild zvType time = 0.68's, 0.337'MB/sec write SST data time = 0.13's, 276.935'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35331.987'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 84.845'MB/sec zip pipeline throughput = 84.845'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.90 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.90 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6989 dict = 1.00 all = 3.8044 } Zip/UnZip{ index = 0.0957 value = 0.2704 dict = 1.00 all = 0.2629 } ---------------------------- total value len = 2253.046357 GB avg = 0.199 KB (by entry num) total key len = 307.272941 GB avg = 0.027 KB total ukey len = 237.633283 GB avg = 0.015 KB total ukey num = 15.752588954 Billion total entry num = 11.293902902 Billion write speed all = 30.389183587 MB/sec (with seq num) write speed all = 29.316778316 MB/sec (without seq num) 2024/11/24-08:19:23.265725 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297423: 919300 keys, 35279218 bytes, temperature: kWarm 2024/11/24-08:19:23.265794 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407563265758, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297423, "file_size": 35279218, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4310314631, "largest_seqno": 4428985405, "table_properties": {"data_size": 29327168, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407560, "oldest_key_time": 0, "file_creation_time": 1732407560, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297423, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:23.304522 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297424.sst) /dev/shm/ToplingTemp/Topling-304735-Of5olQ 2024/11/24-08:19:24.284640 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297424.sst): first pass time = 0.98's, 136.946'MB/sec 2024/11/24-08:19:25.581667 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297424.sst): old prealloc_size = 147639500, real_size = 30468224 2024/11/24-08:19:25.582255 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297424.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928423 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0285 GB avg-val = 116.57 avg-zval = 30.65 2024/11/24-08:19:25.584008 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297424.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:26.398994 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297424.sst): second pass time = 1.30's, 83.489'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.155'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3151.319'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.337'MB/sec write SST data time = 0.13's, 268.696'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35596.109'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.489'MB/sec zip pipeline throughput = 83.489'MB/sec entries = 928423 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.57 avg-zval = 30.65 usrkeys = 928423 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.57 avg-zval = 30.65 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928423 bytes = 3133440 } seq expand size = 6498961 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0285 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.4424 value = 3.8029 dict = 1.00 all = 3.8957 } Zip/UnZip{ index = 0.0958 value = 0.2630 dict = 1.00 all = 0.2567 } ---------------------------- total value len = 2253.161078 GB avg = 0.199 KB (by entry num) total key len = 307.298936 GB avg = 0.027 KB total ukey len = 237.651852 GB avg = 0.015 KB total ukey num = 15.753517377 Billion total entry num = 11.294831325 Billion write speed all = 30.389707463 MB/sec (with seq num) write speed all = 29.317254489 MB/sec (without seq num) 2024/11/24-08:19:26.443022 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297424: 928423 keys, 34453214 bytes, temperature: kWarm 2024/11/24-08:19:26.443090 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407566443053, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297424, "file_size": 34453214, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4428985406, "largest_seqno": 4511408667, "table_properties": {"data_size": 28457776, "index_size": 1778176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995844, "raw_average_key_size": 28, "raw_value_size": 108221942, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928423, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407563, "oldest_key_time": 0, "file_creation_time": 1732407563, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297424, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:26.479068 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297425.sst) /dev/shm/ToplingTemp/Topling-304735-x449wN 2024/11/24-08:19:27.522135 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297425.sst): first pass time = 1.04's, 128.680'MB/sec 2024/11/24-08:19:28.870179 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297425.sst): old prealloc_size = 147639500, real_size = 32924400 2024/11/24-08:19:28.870751 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297425.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930256 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0309 GB avg-val = 116.28 avg-zval = 33.24 2024/11/24-08:19:28.872358 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297425.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:29.734716 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297425.sst): second pass time = 1.35's, 80.324'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.836'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3250.374'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.316'MB/sec write SST data time = 0.13's, 289.192'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25665.485'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.324'MB/sec zip pipeline throughput = 80.324'MB/sec entries = 930256 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.28 avg-zval = 33.24 usrkeys = 930256 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.28 avg-zval = 33.24 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 930256 bytes = 3255904 } seq expand size = 6511792 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5141 value = 3.4982 dict = 1.00 all = 3.6244 } Zip/UnZip{ index = 0.0951 value = 0.2859 dict = 1.00 all = 0.2759 } ---------------------------- total value len = 2253.275760 GB avg = 0.199 KB (by entry num) total key len = 307.324984 GB avg = 0.027 KB total ukey len = 237.670457 GB avg = 0.015 KB total ukey num = 15.754447633 Billion total entry num = 11.295761581 Billion write speed all = 30.390174575 MB/sec (with seq num) write speed all = 29.317675734 MB/sec (without seq num) 2024/11/24-08:19:29.781962 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297425: 930256 keys, 37032054 bytes, temperature: kWarm 2024/11/24-08:19:29.782052 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407569782016, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297425, "file_size": 37032054, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679880, "largest_seqno": 4511407501, "table_properties": {"data_size": 30922144, "index_size": 1769536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26047168, "raw_average_key_size": 28, "raw_value_size": 108170604, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930256, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407566, "oldest_key_time": 0, "file_creation_time": 1732407566, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297425, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:29.818314 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297426.sst) /dev/shm/ToplingTemp/Topling-304735-oYXTER 2024/11/24-08:19:30.815528 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297426.sst): first pass time = 1.00's, 134.598'MB/sec 2024/11/24-08:19:32.141701 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297426.sst): old prealloc_size = 147639500, real_size = 31368432 2024/11/24-08:19:32.142282 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297426.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.94 2024/11/24-08:19:32.143036 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297426.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:32.958280 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297426.sst): second pass time = 1.32's, 81.873'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.268'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3158.533'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 279.689'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 35138.610'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.873'MB/sec zip pipeline throughput = 81.873'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 31.94 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 31.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.3772 value = 3.6939 dict = 1.00 all = 3.7989 } Zip/UnZip{ index = 0.0964 value = 0.2707 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 2253.390673 GB avg = 0.199 KB (by entry num) total key len = 307.350724 GB avg = 0.027 KB total ukey len = 237.688843 GB avg = 0.015 KB total ukey num = 15.755366933 Billion total entry num = 11.296680881 Billion write speed all = 30.390681195 MB/sec (with seq num) write speed all = 29.318136103 MB/sec (without seq num) 2024/11/24-08:19:33.002667 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297426: 919300 keys, 35330354 bytes, temperature: kWarm 2024/11/24-08:19:33.002736 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407573002701, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297426, "file_size": 35330354, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4327643862, "largest_seqno": 4439870031, "table_properties": {"data_size": 29366624, "index_size": 1771776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407569, "oldest_key_time": 0, "file_creation_time": 1732407569, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297426, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:33.039630 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297427.sst) /dev/shm/ToplingTemp/Topling-304735-XrvTYN 2024/11/24-08:19:34.019773 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297427.sst): first pass time = 0.98's, 136.942'MB/sec 2024/11/24-08:19:35.334133 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297427.sst): old prealloc_size = 147639500, real_size = 30415744 2024/11/24-08:19:35.334712 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297427.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929239 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0284 GB avg-val = 116.44 avg-zval = 30.57 2024/11/24-08:19:35.335507 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297427.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:36.156684 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297427.sst): second pass time = 1.31's, 82.374'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 41.826'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3199.517'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.335'MB/sec write SST data time = 0.13's, 267.082'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 29998.110'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.374'MB/sec zip pipeline throughput = 82.374'MB/sec entries = 929239 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.44 avg-zval = 30.57 usrkeys = 929239 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.44 avg-zval = 30.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929239 bytes = 3136192 } seq expand size = 6504673 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0344 GB } UnZip/Zip{ index = 10.4445 value = 3.8093 dict = 1.00 all = 3.9013 } Zip/UnZip{ index = 0.0957 value = 0.2625 dict = 1.00 all = 0.2563 } ---------------------------- total value len = 2253.505376 GB avg = 0.199 KB (by entry num) total key len = 307.376743 GB avg = 0.027 KB total ukey len = 237.707428 GB avg = 0.015 KB total ukey num = 15.756296172 Billion total entry num = 11.297610120 Billion write speed all = 30.391197677 MB/sec (with seq num) write speed all = 29.318605074 MB/sec (without seq num) 2024/11/24-08:19:36.201679 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297427: 929239 keys, 34402996 bytes, temperature: kWarm 2024/11/24-08:19:36.201753 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407576201716, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297427, "file_size": 34402996, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4439870032, "largest_seqno": 4511408678, "table_properties": {"data_size": 28403888, "index_size": 1779392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26018692, "raw_average_key_size": 28, "raw_value_size": 108199090, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929239, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407573, "oldest_key_time": 0, "file_creation_time": 1732407573, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297427, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:36.248305 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297428.sst) /dev/shm/ToplingTemp/Topling-304735-ajaXvQ 2024/11/24-08:19:37.246006 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297428.sst): first pass time = 1.00's, 134.533'MB/sec 2024/11/24-08:19:38.569190 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297428.sst): old prealloc_size = 147639500, real_size = 32864368 2024/11/24-08:19:38.569762 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297428.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924460 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.19 avg-zval = 33.40 2024/11/24-08:19:38.571375 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297428.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:39.415941 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297428.sst): second pass time = 1.32's, 81.924'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.85's, 43.646'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3214.007'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.322'MB/sec write SST data time = 0.13's, 285.531'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32729.749'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.924'MB/sec zip pipeline throughput = 81.924'MB/sec entries = 924460 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.19 avg-zval = 33.40 usrkeys = 924460 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.19 avg-zval = 33.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924460 bytes = 3235632 } seq expand size = 6471220 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.5136 value = 3.5088 dict = 1.00 all = 3.6318 } Zip/UnZip{ index = 0.0951 value = 0.2850 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 2253.620181 GB avg = 0.199 KB (by entry num) total key len = 307.402628 GB avg = 0.027 KB total ukey len = 237.725917 GB avg = 0.015 KB total ukey num = 15.757220632 Billion total entry num = 11.298534580 Billion write speed all = 30.391691774 MB/sec (with seq num) write speed all = 29.319052891 MB/sec (without seq num) 2024/11/24-08:19:39.466306 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297428: 924460 keys, 36955964 bytes, temperature: kWarm 2024/11/24-08:19:39.466378 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407579466341, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297428, "file_size": 36955964, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679891, "largest_seqno": 4511409694, "table_properties": {"data_size": 30874464, "index_size": 1758592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25884880, "raw_average_key_size": 28, "raw_value_size": 108332900, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924460, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407576, "oldest_key_time": 0, "file_creation_time": 1732407576, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297428, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:39.503111 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297429.sst) /dev/shm/ToplingTemp/Topling-304735-YlWq5P 2024/11/24-08:19:40.402174 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297429.sst): first pass time = 0.90's, 149.293'MB/sec 2024/11/24-08:19:41.710404 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297429.sst): old prealloc_size = 147639500, real_size = 31337232 2024/11/24-08:19:41.710951 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297429.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.92 2024/11/24-08:19:41.711702 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297429.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:42.526446 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297429.sst): second pass time = 1.31's, 82.996'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.258'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3338.648'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.333'MB/sec write SST data time = 0.13's, 280.056'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 31600.699'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.996'MB/sec zip pipeline throughput = 82.996'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.92 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6964 dict = 1.00 all = 3.8023 } Zip/UnZip{ index = 0.0957 value = 0.2705 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 2253.735093 GB avg = 0.199 KB (by entry num) total key len = 307.428368 GB avg = 0.027 KB total ukey len = 237.744303 GB avg = 0.015 KB total ukey num = 15.758139932 Billion total entry num = 11.299453880 Billion write speed all = 30.392239054 MB/sec (with seq num) write speed all = 29.319552492 MB/sec (without seq num) 2024/11/24-08:19:42.570684 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297429: 919300 keys, 35299058 bytes, temperature: kWarm 2024/11/24-08:19:42.570754 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407582570718, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297429, "file_size": 35299058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4351432748, "largest_seqno": 4463510049, "table_properties": {"data_size": 29347008, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407579, "oldest_key_time": 0, "file_creation_time": 1732407579, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297429, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:42.605732 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297430.sst) /dev/shm/ToplingTemp/Topling-304735-20i2tP 2024/11/24-08:19:43.440006 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297430.sst): first pass time = 0.83's, 160.886'MB/sec 2024/11/24-08:19:44.755614 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297430.sst): old prealloc_size = 147639500, real_size = 30406560 2024/11/24-08:19:44.756226 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297430.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933122 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0284 GB avg-val = 115.84 avg-zval = 30.43 2024/11/24-08:19:44.757004 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297430.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:45.583470 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297430.sst): second pass time = 1.31's, 82.227'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.83's, 41.703'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3022.748'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.333'MB/sec write SST data time = 0.13's, 270.624'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 35902.018'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.227'MB/sec zip pipeline throughput = 82.227'MB/sec entries = 933122 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.84 avg-zval = 30.43 usrkeys = 933122 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.84 avg-zval = 30.43 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933122 bytes = 3265936 } seq expand size = 6531854 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0284 GB dict = 1.08 MB all = 0.0345 GB } UnZip/Zip{ index = 10.5142 value = 3.8063 dict = 1.00 all = 3.8878 } Zip/UnZip{ index = 0.0951 value = 0.2627 dict = 1.00 all = 0.2572 } ---------------------------- total value len = 2253.849715 GB avg = 0.199 KB (by entry num) total key len = 307.454495 GB avg = 0.027 KB total ukey len = 237.762965 GB avg = 0.015 KB total ukey num = 15.759073054 Billion total entry num = 11.300387002 Billion write speed all = 30.392806729 MB/sec (with seq num) write speed all = 29.320070498 MB/sec (without seq num) 2024/11/24-08:19:45.628464 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297430: 933122 keys, 34523066 bytes, temperature: kWarm 2024/11/24-08:19:45.628541 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407585628502, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297430, "file_size": 34523066, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679902, "largest_seqno": 4511410048, "table_properties": {"data_size": 28398096, "index_size": 1774976, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26127416, "raw_average_key_size": 28, "raw_value_size": 108090336, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933122, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407582, "oldest_key_time": 0, "file_creation_time": 1732407582, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297430, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:45.664696 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297431.sst) /dev/shm/ToplingTemp/Topling-304735-BuFAoR 2024/11/24-08:19:46.526374 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297431.sst): first pass time = 0.86's, 155.772'MB/sec 2024/11/24-08:19:47.803094 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297431.sst): old prealloc_size = 147639500, real_size = 31333776 2024/11/24-08:19:47.803665 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297431.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.92 2024/11/24-08:19:47.804456 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297431.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:48.627780 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297431.sst): second pass time = 1.28's, 85.035'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.800'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3214.657'MB/sec (index lex order gen) rebuild zvType time = 0.70's, 0.330'MB/sec write SST data time = 0.13's, 275.188'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 29230.583'MB/sec zip my value time = 1.28's, unzip length = 0.108'GB zip my value throughput = 85.035'MB/sec zip pipeline throughput = 85.035'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.92 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6968 dict = 1.00 all = 3.8027 } Zip/UnZip{ index = 0.0957 value = 0.2705 dict = 1.00 all = 0.2630 } ---------------------------- total value len = 2253.964628 GB avg = 0.199 KB (by entry num) total key len = 307.480236 GB avg = 0.027 KB total ukey len = 237.781351 GB avg = 0.015 KB total ukey num = 15.759992354 Billion total entry num = 11.301306302 Billion write speed all = 30.393377800 MB/sec (with seq num) write speed all = 29.320593055 MB/sec (without seq num) 2024/11/24-08:19:48.672584 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297431: 919300 keys, 35295602 bytes, temperature: kWarm 2024/11/24-08:19:48.672658 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407588672621, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297431, "file_size": 35295602, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4270786263, "largest_seqno": 4378944863, "table_properties": {"data_size": 29343552, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407585, "oldest_key_time": 0, "file_creation_time": 1732407585, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297431, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:48.708403 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297432.sst) /dev/shm/ToplingTemp/Topling-304735-8m12mO 2024/11/24-08:19:49.697831 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297432.sst): first pass time = 0.99's, 135.657'MB/sec 2024/11/24-08:19:51.005335 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297432.sst): old prealloc_size = 147639500, real_size = 31360080 2024/11/24-08:19:51.005879 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297432.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0294 GB avg-val = 118.00 avg-zval = 31.95 2024/11/24-08:19:51.006707 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297432.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:51.826738 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297432.sst): second pass time = 1.31's, 83.017'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 43.003'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3382.981'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 274.840'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 33874.805'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 83.017'MB/sec zip pipeline throughput = 83.017'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.95 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4455 value = 3.6935 dict = 1.00 all = 3.7998 } Zip/UnZip{ index = 0.0957 value = 0.2707 dict = 1.00 all = 0.2632 } ---------------------------- total value len = 2254.079540 GB avg = 0.199 KB (by entry num) total key len = 307.505976 GB avg = 0.027 KB total ukey len = 237.799737 GB avg = 0.015 KB total ukey num = 15.760911654 Billion total entry num = 11.302225602 Billion write speed all = 30.393893057 MB/sec (with seq num) write speed all = 29.321061769 MB/sec (without seq num) 2024/11/24-08:19:51.871380 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297432: 919300 keys, 35321906 bytes, temperature: kWarm 2024/11/24-08:19:51.871447 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407591871411, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297432, "file_size": 35321906, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4378944864, "largest_seqno": 4491749903, "table_properties": {"data_size": 29369856, "index_size": 1760192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407588, "oldest_key_time": 0, "file_creation_time": 1732407588, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297432, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:51.911267 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297433.sst) /dev/shm/ToplingTemp/Topling-304735-qczSkP 2024/11/24-08:19:52.897990 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297433.sst): first pass time = 0.99's, 136.032'MB/sec 2024/11/24-08:19:54.248948 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297433.sst): old prealloc_size = 147639500, real_size = 32248384 2024/11/24-08:19:54.249529 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297433.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 933139 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0302 GB avg-val = 115.83 avg-zval = 32.41 2024/11/24-08:19:54.251617 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297433.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:55.109217 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297433.sst): second pass time = 1.35's, 80.086'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.273'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3349.249'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.321'MB/sec write SST data time = 0.13's, 274.042'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 34114.177'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.086'MB/sec zip pipeline throughput = 80.086'MB/sec entries = 933139 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.83 avg-zval = 32.41 usrkeys = 933139 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.83 avg-zval = 32.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 933139 bytes = 3266000 } seq expand size = 6531973 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5155 value = 3.5744 dict = 1.00 all = 3.6908 } Zip/UnZip{ index = 0.0951 value = 0.2798 dict = 1.00 all = 0.2709 } ---------------------------- total value len = 2254.194162 GB avg = 0.199 KB (by entry num) total key len = 307.532104 GB avg = 0.027 KB total ukey len = 237.818400 GB avg = 0.015 KB total ukey num = 15.761844793 Billion total entry num = 11.303158741 Billion write speed all = 30.394379306 MB/sec (with seq num) write speed all = 29.321501228 MB/sec (without seq num) 2024/11/24-08:19:55.155371 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297433: 933139 keys, 36365000 bytes, temperature: kWarm 2024/11/24-08:19:55.155443 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407595155405, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297433, "file_size": 36365000, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679921, "largest_seqno": 4511409180, "table_properties": {"data_size": 30240112, "index_size": 1774784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26127892, "raw_average_key_size": 28, "raw_value_size": 108089862, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 933139, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407591, "oldest_key_time": 0, "file_creation_time": 1732407591, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297433, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:55.193461 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297434.sst) /dev/shm/ToplingTemp/Topling-304735-M8nVmQ 2024/11/24-08:19:56.187411 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297434.sst): first pass time = 0.99's, 135.040'MB/sec 2024/11/24-08:19:57.496337 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297434.sst): old prealloc_size = 147639500, real_size = 31324752 2024/11/24-08:19:57.496865 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297434.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919300 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0293 GB avg-val = 118.00 avg-zval = 31.91 2024/11/24-08:19:57.498854 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297434.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:19:58.319689 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297434.sst): second pass time = 1.31's, 82.953'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.859'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3504.596'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 268.274'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34440.389'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.953'MB/sec zip pipeline throughput = 82.953'MB/sec entries = 919300 avg-key = 28.00 avg-zkey = 1.91 avg-val = 118.00 avg-zval = 31.91 usrkeys = 919300 avg-key = 20.00 avg-zkey = 1.91 avg-val = 125.00 avg-zval = 31.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 919300 bytes = 3102656 } seq expand size = 6435100 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0293 GB dict = 1.09 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4447 value = 3.6980 dict = 1.00 all = 3.8037 } Zip/UnZip{ index = 0.0957 value = 0.2704 dict = 1.00 all = 0.2629 } ---------------------------- total value len = 2254.309075 GB avg = 0.199 KB (by entry num) total key len = 307.557845 GB avg = 0.027 KB total ukey len = 237.836786 GB avg = 0.015 KB total ukey num = 15.762764093 Billion total entry num = 11.304078041 Billion write speed all = 30.394890334 MB/sec (with seq num) write speed all = 29.321965867 MB/sec (without seq num) 2024/11/24-08:19:58.364333 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297434: 919300 keys, 35286578 bytes, temperature: kWarm 2024/11/24-08:19:58.364407 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407598364369, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297434, "file_size": 35286578, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4296693280, "largest_seqno": 4406739109, "table_properties": {"data_size": 29334400, "index_size": 1760320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25740400, "raw_average_key_size": 28, "raw_value_size": 108477400, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 919300, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407595, "oldest_key_time": 0, "file_creation_time": 1732407595, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297434, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:19:58.403743 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297435.sst) /dev/shm/ToplingTemp/Topling-304735-k9lKyQ 2024/11/24-08:19:59.416893 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297435.sst): first pass time = 1.01's, 132.481'MB/sec 2024/11/24-08:20:00.738669 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297435.sst): old prealloc_size = 147639500, real_size = 31232496 2024/11/24-08:20:00.739305 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297435.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 921357 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0292 GB avg-val = 117.67 avg-zval = 31.73 2024/11/24-08:20:00.740079 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297435.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/24-08:20:01.562597 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297435.sst): second pass time = 1.32's, 82.072'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.82's, 42.723'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3008.485'MB/sec (index lex order gen) rebuild zvType time = 0.69's, 0.332'MB/sec write SST data time = 0.13's, 269.764'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 32035.468'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.072'MB/sec zip pipeline throughput = 82.072'MB/sec entries = 921357 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.67 avg-zval = 31.73 usrkeys = 921357 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.67 avg-zval = 31.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 921357 bytes = 3109600 } seq expand size = 6449499 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0184 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0352 GB } UnZip/Zip{ index = 10.4438 value = 3.7082 dict = 1.00 all = 3.8130 } Zip/UnZip{ index = 0.0958 value = 0.2697 dict = 1.00 all = 0.2623 } ---------------------------- total value len = 2254.423944 GB avg = 0.199 KB (by entry num) total key len = 307.583643 GB avg = 0.027 KB total ukey len = 237.855213 GB avg = 0.015 KB total ukey num = 15.763685450 Billion total entry num = 11.304999398 Billion write speed all = 30.395389798 MB/sec (with seq num) write speed all = 29.322419163 MB/sec (without seq num) 2024/11/24-08:20:01.606988 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297435: 921357 keys, 35199584 bytes, temperature: kWarm 2024/11/24-08:20:01.607063 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407601607025, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297435, "file_size": 35199584, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4406739110, "largest_seqno": 4511408731, "table_properties": {"data_size": 29237536, "index_size": 1764416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25797996, "raw_average_key_size": 28, "raw_value_size": 108419746, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 921357, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407598, "oldest_key_time": 0, "file_creation_time": 1732407598, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297435, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:20:01.647152 139957492123200 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297436.sst) /dev/shm/ToplingTemp/Topling-304735-2WE8BO 2024/11/24-08:20:01.999828 139957492123200 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297436.sst): first pass time = 0.35's, 242.699'MB/sec 2024/11/24-08:20:02.849645 139957492123200 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297436.sst): old prealloc_size = 147639500, real_size = 21316592 2024/11/24-08:20:02.850101 139957492123200 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297436.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 597953 prefix = 4 raw-key = 0.0096 GB zip-key = 0.0011 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0688 GB zip-val = 0.0200 GB avg-val = 115.13 avg-zval = 33.48 2024/11/24-08:20:02.850857 139957492123200 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297436.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.006 GB, reorder workingMem = 0.0015 GB, waited 0.000 sec, Key+Value = 0.086 GB 2024/11/24-08:20:03.395603 139957492123200 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00/297436.sst): second pass time = 0.85's, 81.086'MB/sec, value only(80.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.55's, 43.876'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2649.491'MB/sec (index lex order gen) rebuild zvType time = 0.46's, 0.324'MB/sec write SST data time = 0.08's, 286.757'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.690'MB, throughput = 18874.836'MB/sec zip my value time = 0.85's, unzip length = 0.069'GB zip my value throughput = 81.086'MB/sec zip pipeline throughput = 81.086'MB/sec entries = 597953 avg-key = 28.00 avg-zkey = 1.92 avg-val = 115.13 avg-zval = 33.48 usrkeys = 597953 avg-key = 20.00 avg-zkey = 1.92 avg-val = 122.13 avg-zval = 33.48 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 597953 bytes = 2092848 } seq expand size = 4185671 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0120 GB value = 0.0688 GB dict = 0.69 MB all = 0.0856 GB } __ZipSize{ index = 0.0011 GB value = 0.0200 GB dict = 0.69 MB all = 0.0240 GB } UnZip/Zip{ index = 10.4391 value = 3.4383 dict = 1.00 all = 3.5728 } Zip/UnZip{ index = 0.0958 value = 0.2908 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 2254.496969 GB avg = 0.199 KB (by entry num) total key len = 307.600385 GB avg = 0.027 KB total ukey len = 237.867172 GB avg = 0.015 KB total ukey num = 15.764283403 Billion total entry num = 11.305597351 Billion write speed all = 30.395793788 MB/sec (with seq num) write speed all = 29.322789735 MB/sec (without seq num) 2024/11/24-08:20:03.427513 139957492123200 [compaction_job.cc:1948] [default] [JOB 43894] Generated table #297436: 597953 keys, 23953905 bytes, temperature: kWarm 2024/11/24-08:20:03.427581 139957492123200 EVENT_LOG_v1 {"time_micros": 1732407603427546, "cf_name": "default", "job": 43894, "event": "table_file_creation", "file_number": 297436, "file_size": 23953905, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4265679941, "largest_seqno": 4511407388, "table_properties": {"data_size": 20021344, "index_size": 1145600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 16742684, "raw_average_key_size": 28, "raw_value_size": 68839682, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 597953, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732407601, "oldest_key_time": 0, "file_creation_time": 1732407601, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 297436, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/24-08:20:03.449018 139957492123200 [compaction_job.cc:703] job-43894: subcompact[0], size: 0.983640 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-43894/att-00 [297409,297410,297411,297412,297413,297414,297415,297416,297417,297418,297419,297420,297421,297422,297423,297424,297425,297426,297427,297428,297429,297430,297431,297432,297433,297434,297435,297436] 2024/11/24-08:20:03.618330 139957492123200 (Original Log Time 2024/11/24-08:18:34.552854) EVENT_LOG_v1 {"time_micros": 1732407514552811, "job": 43894, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [297372], "files_L3": [297386, 297387, 297314, 297315, 297316, 297317, 297318, 297319, 297320, 297321, 297322, 297323, 297324, 297325, 297326, 297327, 297328, 297329, 297330, 297331, 297332, 297333, 297334, 297335, 297336, 297337, 297298, 297299, 297397], "score": 0.796637, "input_data_size": 1002477827, "oldest_snapshot_seqno": 4511594776} 2024/11/24-08:20:03.618340 139957492123200 (Original Log Time 2024/11/24-08:20:03.616345) [compaction_job.cc:2022] [default] [JOB 43894] Compacted 1@2 + 29@3 files to L3 => 983640073 bytes 2024/11/24-08:20:03.618341 139957492123200 (Original Log Time 2024/11/24-08:20:03.618253) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 28 0 0 0] max score 0.04, MB/sec: 11.3 rd, 11.1 wr, level 3, files in(1, 29) out(28 +0 blob) MB in(27.1, 929.0 +0.0 blob) out(938.1 +0.0 blob), read-write-amplify(70.0) write-amplify(34.7) OK, records in: 25715371, records dropped: 93131 output_compression: Snappy 2024/11/24-08:20:03.618345 139957492123200 (Original Log Time 2024/11/24-08:20:03.618302) EVENT_LOG_v1 {"time_micros": 1732407603618277, "cf": "default", "job": 43894, "event": "compaction_finished", "compaction_time_micros": 88896174, "compaction_time_cpu_micros": 38686601, "output_level": 3, "num_output_files": 28, "total_output_size": 983640073, "num_input_records": 25715371, "num_output_records": 25622240, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 687330023, "file_range_sync_nanos": 0, "file_fsync_nanos": 18279, "file_prepare_write_nanos": 73845, "lsm_state": [0, 0, 0, 28, 0, 0, 0]} 2024/11/24-08:20:03.771548 139957492123200 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732407514,"executesMs":89190,"compactionJobId":43894,"attempt":0,"compactionInputRawBytes":3722687622,"compactionInputZipBytes":1002477827,"compactionOutputRawBytes":0,"compactionOutputZipBytes":983640073,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-dee9d131f61bdc6ce9c82f22c9deb958-9fc5fdd4e434a2e5-00"}}, response = 2024/11/24-08:20:03.771630 139957492123200 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-43894/att-00: olev 3, work 89.190 s, result 5.916 ms, install 2.016 ms, input{raw 3.467 GiB zip 956.037 MiB}