2024/07/08-17:32:53.365758 140554168411712 DEBG dcompact_worker.cpp:2211: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00: fork to child time = 0.008639 sec 2024/07/08-17:32:53.366465 140554168411712 DEBG dcompact_worker.cpp:1011: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00 2024/07/08-17:32:53.373194 140554168411712 DEBG dcompact_worker.cpp:1031: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00 2024/07/08-17:32:53.374555 140554168411712 TRAC dcompact_worker.cpp:1081: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/07/08-17:32:53.374870 140554168411712 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/07/08-17:32:53.415813 140554168411712 [version_set.cc:6139] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-08639/att-00/MANIFEST-12625661 2024/07/08-17:32:53.500677 140554168411712 [version_set.cc:6182] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-08639/att-00/MANIFEST-12625661 succeeded,manifest_file_number is 12625661, next_file_number is 12634215, last_sequence is 147361294327, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 12634103 2024/07/08-17:32:53.500696 140554168411712 [version_set.cc:6197] Column family [default] (ID 0), log number is 0 2024/07/08-17:32:53.500929 140554168411712 [version_set.cc:5652] Creating manifest 12634215 2024/07/08-17:32:53.741318 140554168411712 DEBG dcompact_worker.cpp:1215: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00: bottommost_level: fake = 1, rpc = 1 2024/07/08-17:32:53.763482 140554168411712 [compaction_job.cc:2433] [default] [JOB 8639] Compacting 20@5 + 27@6 files to L6, score -1.00, subcompactions 1 : 1 2024/07/08-17:32:53.763520 140554168411712 [compaction_job.cc:2441] [default]: Compaction start summary: Base version 4881 Base level 5, inputs: [12628932(116MB) 12628936(136MB) 12628937(119MB) 12628938(136MB) 12628939(121MB) 12628940(136MB) 12628941(122MB) 12628942(136MB) 12628943(122MB) 12628944(136MB) 12628945(122MB) 12628946(136MB) 12628947(122MB) 12628948(136MB) 12628949(120MB) 12628950(136MB) 12628951(119MB) 12628952(128MB) 12629150(35MB) 12629151(9256KB)], [12634212(222MB) 12634213(118MB) 12393294(187MB) 12393295(175MB) 12393296(222MB) 12393297(185MB) 12393298(190MB) 12393299(191MB) 12393300(190MB) 12393301(185MB) 12393302(220MB) 12393303(178MB) 12393304(187MB) 12393305(237MB) 12393306(220MB) 12393307(178MB) 12393308(189MB) 12393309(191MB) 12393310(183MB) 12393311(222MB) 12393312(184MB) 12393313(191MB) 12393314(189MB) 12393315(175MB) 12393316(221MB) 12393317(188MB) 12393318(217MB)] 2024/07/08-17:32:53.769000 140554168411712 [top_zip_table.cc:59] core git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-17:32:53.769042 140554168411712 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-17:32:53.769081 140554168411712 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit 7517bad5c1a147474c3830831b9a22c996321ae8 Author: leipeng Date: 2024-01-08 19:18:17 +0800 vfork_cmd: call close before wait_finish g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/07/08-17:32:53.769103 140554168411712 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit 7f80399bfbd2071b6e0f5683e8ab7225b8e6e394 Author: leipeng Date: 2024-01-19 14:52:30 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/07/08-17:32:53.769113 140554168411712 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit 1240987ca13d0fe562c9b9c037155079f0877840 Author: leipeng Date: 2024-01-19 15:07:04 +0800 Change ToplingZipTableOptions::sampleRatio to vector g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/07/08-17:32:56.594059 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634216.sst) /dev/shm/ToplingTemp/Topling-17573-iS4qRY 2024/07/08-17:33:01.523762 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634216.sst): first pass time = 4.93's, 217.814'MB/sec 2024/07/08-17:33:10.429718 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634216.sst): old prealloc_size = 1073741824, real_size = 205220912 2024/07/08-17:33:10.433982 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634216.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7388456 prefix = 4 raw-key = 0.1182 GB zip-key = 0.0141 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.8669 GB zip-val = 0.1892 GB avg-val = 117.33 avg-zval = 25.61 2024/07/08-17:33:10.434049 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634216.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:33:15.996603 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634216.sst): second pass time = 8.90's, 97.369'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.57's, 38.104'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3366.470'MB/sec (index lex order gen) rebuild zvType time = 4.58's, 0.404'MB/sec write SST data time = 0.99's, 215.104'MB/sec dict compress time = 0.00's, 8680410.000'MB/sec z-dict build time = 0.00's, sample length = 8.680'MB, throughput = 156821.976'MB/sec zip my value time = 8.90's, unzip length = 0.867'GB zip my value throughput = 97.369'MB/sec zip pipeline throughput = 97.607'MB/sec entries = 7388456 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.33 avg-zval = 25.61 usrkeys = 7388456 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.33 avg-zval = 25.61 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.1478 GB value = 0.8669 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0141 GB value = 0.1892 GB dict = 8.68 MB all = 0.2121 GB } UnZip/Zip{ index = 10.4521 value = 4.5809 dict = 1.00 all = 5.0619 } Zip/UnZip{ index = 0.0957 value = 0.2183 dict = 1.00 all = 0.1976 } ---------------------------- total value len = 0.866865 GB avg = 0.117 KB (by entry num) total key len = 0.206877 GB avg = 0.028 KB total ukey len = 0.147769 GB avg = 0.020 KB total ukey num = 0.007388456 Billion total entry num = 0.007388456 Billion write speed all = 0.421921726 MB/sec (with seq num) write speed all = 0.398695659 MB/sec (without seq num) 2024/07/08-17:33:16.228276 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634216: 7388456 keys, 212121889 bytes, temperature: kWarm 2024/07/08-17:33:16.228360 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431196228313, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634216, "file_size": 212121889, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 189235872, "index_size": 14137728, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206876768, "raw_average_key_size": 28, "raw_value_size": 866865128, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7388456, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431176, "oldest_key_time": 0, "file_creation_time": 1720431176, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634216, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:33:16.370768 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634217.sst) /dev/shm/ToplingTemp/Topling-17573-hr6yP0 2024/07/08-17:33:22.167401 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634217.sst): first pass time = 5.80's, 185.237'MB/sec 2024/07/08-17:33:30.333348 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634217.sst): old prealloc_size = 1073741824, real_size = 193041872 2024/07/08-17:33:30.337683 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634217.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7374013 prefix = 4 raw-key = 0.1180 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8673 GB zip-val = 0.1772 GB avg-val = 117.61 avg-zval = 24.03 2024/07/08-17:33:30.337743 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634217.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:33:35.926924 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634217.sst): second pass time = 8.16's, 106.238'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.59's, 35.741'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3265.179'MB/sec (index lex order gen) rebuild zvType time = 4.60's, 0.401'MB/sec write SST data time = 0.99's, 202.000'MB/sec dict compress time = 0.00's, 8684110.000'MB/sec z-dict build time = 0.00's, sample length = 8.684'MB, throughput = 162292.512'MB/sec zip my value time = 8.16's, unzip length = 0.867'GB zip my value throughput = 106.238'MB/sec zip pipeline throughput = 106.238'MB/sec entries = 7374013 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.61 avg-zval = 24.03 usrkeys = 7374013 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.61 avg-zval = 24.03 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1475 GB value = 0.8673 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1772 GB dict = 8.68 MB all = 0.1999 GB } UnZip/Zip{ index = 10.5205 value = 4.8949 dict = 1.00 all = 5.3708 } Zip/UnZip{ index = 0.0951 value = 0.2043 dict = 1.00 all = 0.1862 } ---------------------------- total value len = 1.734135 GB avg = 0.117 KB (by entry num) total key len = 0.413349 GB avg = 0.028 KB total ukey len = 0.295249 GB avg = 0.020 KB total ukey num = 0.014762469 Billion total entry num = 0.014762469 Billion write speed all = 0.837286236 MB/sec (with seq num) write speed all = 0.791240116 MB/sec (without seq num) 2024/07/08-17:33:36.146197 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634217: 7374013 keys, 199920533 bytes, temperature: kWarm 2024/07/08-17:33:36.146268 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431216146232, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634217, "file_size": 199920533, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 177179904, "index_size": 14018304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206472364, "raw_average_key_size": 28, "raw_value_size": 867269562, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7374013, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431196, "oldest_key_time": 0, "file_creation_time": 1720431196, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634217, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:33:36.290934 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634218.sst) /dev/shm/ToplingTemp/Topling-17573-4Ii490 2024/07/08-17:33:42.124049 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634218.sst): first pass time = 5.83's, 184.078'MB/sec 2024/07/08-17:33:50.191551 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634218.sst): old prealloc_size = 1073741824, real_size = 191991952 2024/07/08-17:33:50.195769 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634218.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7373887 prefix = 4 raw-key = 0.1180 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8673 GB zip-val = 0.1761 GB avg-val = 117.61 avg-zval = 23.89 2024/07/08-17:33:50.195847 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634218.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:33:55.786425 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634218.sst): second pass time = 8.06's, 107.537'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.59's, 35.545'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3347.604'MB/sec (index lex order gen) rebuild zvType time = 4.60's, 0.401'MB/sec write SST data time = 0.99's, 200.549'MB/sec dict compress time = 0.00's, 8684076.000'MB/sec z-dict build time = 0.00's, sample length = 8.684'MB, throughput = 175545.817'MB/sec zip my value time = 8.06's, unzip length = 0.867'GB zip my value throughput = 107.537'MB/sec zip pipeline throughput = 107.537'MB/sec entries = 7373887 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.61 avg-zval = 23.89 usrkeys = 7373887 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.61 avg-zval = 23.89 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1475 GB value = 0.8673 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1761 GB dict = 8.68 MB all = 0.1989 GB } UnZip/Zip{ index = 10.5214 value = 4.9240 dict = 1.00 all = 5.3992 } Zip/UnZip{ index = 0.0950 value = 0.2031 dict = 1.00 all = 0.1852 } ---------------------------- total value len = 2.601408 GB avg = 0.118 KB (by entry num) total key len = 0.619818 GB avg = 0.028 KB total ukey len = 0.442727 GB avg = 0.020 KB total ukey num = 0.022136356 Billion total entry num = 0.022136356 Billion write speed all = 1.246279312 MB/sec (with seq num) write speed all = 1.177763571 MB/sec (without seq num) 2024/07/08-17:33:55.999408 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634218: 7373887 keys, 198870579 bytes, temperature: kWarm 2024/07/08-17:33:55.999476 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431235999441, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634218, "file_size": 198870579, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 176131328, "index_size": 14016960, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206468836, "raw_average_key_size": 28, "raw_value_size": 867272998, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7373887, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431216, "oldest_key_time": 0, "file_creation_time": 1720431216, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634218, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:33:56.143046 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634219.sst) /dev/shm/ToplingTemp/Topling-17573-WPjDYX 2024/07/08-17:34:01.996173 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634219.sst): first pass time = 5.85's, 183.449'MB/sec 2024/07/08-17:34:10.034009 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634219.sst): old prealloc_size = 1073741824, real_size = 190656960 2024/07/08-17:34:10.038317 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634219.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7374010 prefix = 4 raw-key = 0.1180 GB zip-key = 0.0142 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.8673 GB zip-val = 0.1746 GB avg-val = 117.61 avg-zval = 23.68 2024/07/08-17:34:10.038371 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634219.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:34:15.638951 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634219.sst): second pass time = 8.04's, 107.935'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.60's, 35.243'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3328.154'MB/sec (index lex order gen) rebuild zvType time = 4.61's, 0.400'MB/sec write SST data time = 0.99's, 198.857'MB/sec dict compress time = 0.00's, 8684606.000'MB/sec z-dict build time = 0.00's, sample length = 8.685'MB, throughput = 182991.761'MB/sec zip my value time = 8.04's, unzip length = 0.867'GB zip my value throughput = 107.935'MB/sec zip pipeline throughput = 107.935'MB/sec entries = 7374010 avg-key = 28.00 avg-zkey = 1.93 avg-val = 117.61 avg-zval = 23.68 usrkeys = 7374010 avg-key = 20.00 avg-zkey = 1.93 avg-val = 117.61 avg-zval = 23.68 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.1475 GB value = 0.8673 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0142 GB value = 0.1746 GB dict = 8.68 MB all = 0.1975 GB } UnZip/Zip{ index = 10.3858 value = 4.9668 dict = 1.00 all = 5.4357 } Zip/UnZip{ index = 0.0963 value = 0.2013 dict = 1.00 all = 0.1840 } ---------------------------- total value len = 3.468677 GB avg = 0.118 KB (by entry num) total key len = 0.826290 GB avg = 0.028 KB total ukey len = 0.590207 GB avg = 0.020 KB total ukey num = 0.029510366 Billion total entry num = 0.029510366 Billion write speed all = 1.649039709 MB/sec (with seq num) write speed all = 1.558396388 MB/sec (without seq num) 2024/07/08-17:34:15.854324 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634219: 7374010 keys, 197536133 bytes, temperature: kWarm 2024/07/08-17:34:15.854393 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431255854358, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634219, "file_size": 197536133, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 174613104, "index_size": 14200192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206472280, "raw_average_key_size": 28, "raw_value_size": 867269620, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7374010, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431236, "oldest_key_time": 0, "file_creation_time": 1720431236, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634219, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:34:16.000995 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634220.sst) /dev/shm/ToplingTemp/Topling-17573-bnHck0 2024/07/08-17:34:21.856050 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634220.sst): first pass time = 5.86's, 183.389'MB/sec 2024/07/08-17:34:30.012925 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634220.sst): old prealloc_size = 1073741824, real_size = 188650848 2024/07/08-17:34:30.017277 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634220.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7374210 prefix = 4 raw-key = 0.1180 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8673 GB zip-val = 0.1728 GB avg-val = 117.61 avg-zval = 23.43 2024/07/08-17:34:30.017348 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634220.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:34:35.613048 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634220.sst): second pass time = 8.15's, 106.356'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.60's, 34.915'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3256.150'MB/sec (index lex order gen) rebuild zvType time = 4.61's, 0.400'MB/sec write SST data time = 0.99's, 197.444'MB/sec dict compress time = 0.00's, 8684658.000'MB/sec z-dict build time = 0.00's, sample length = 8.685'MB, throughput = 159345.675'MB/sec zip my value time = 8.15's, unzip length = 0.867'GB zip my value throughput = 106.356'MB/sec zip pipeline throughput = 106.356'MB/sec entries = 7374210 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.61 avg-zval = 23.43 usrkeys = 7374210 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.61 avg-zval = 23.43 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.1475 GB value = 0.8673 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1728 GB dict = 8.68 MB all = 0.1955 GB } UnZip/Zip{ index = 10.5216 value = 5.0192 dict = 1.00 all = 5.4914 } Zip/UnZip{ index = 0.0950 value = 0.1992 dict = 1.00 all = 0.1821 } ---------------------------- total value len = 4.335941 GB avg = 0.118 KB (by entry num) total key len = 1.032768 GB avg = 0.028 KB total ukey len = 0.737692 GB avg = 0.020 KB total ukey num = 0.036884576 Billion total entry num = 0.036884576 Billion write speed all = 2.045611881 MB/sec (with seq num) write speed all = 1.933180353 MB/sec (without seq num) 2024/07/08-17:34:35.823676 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634220: 7374210 keys, 195529977 bytes, temperature: kWarm 2024/07/08-17:34:35.823752 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431275823717, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634220, "file_size": 195529977, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 172789776, "index_size": 14017344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206477880, "raw_average_key_size": 28, "raw_value_size": 867264084, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7374210, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431255, "oldest_key_time": 0, "file_creation_time": 1720431255, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634220, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:34:35.963821 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634221.sst) /dev/shm/ToplingTemp/Topling-17573-PGGgnX 2024/07/08-17:34:41.829193 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634221.sst): first pass time = 5.87's, 183.066'MB/sec 2024/07/08-17:34:49.958402 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634221.sst): old prealloc_size = 1073741824, real_size = 187170560 2024/07/08-17:34:49.962853 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634221.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7375152 prefix = 4 raw-key = 0.1180 GB zip-key = 0.0141 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.8672 GB zip-val = 0.1712 GB avg-val = 117.59 avg-zval = 23.22 2024/07/08-17:34:49.962902 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634221.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:34:55.572832 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634221.sst): second pass time = 8.13's, 106.717'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.61's, 34.563'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3198.632'MB/sec (index lex order gen) rebuild zvType time = 4.62's, 0.399'MB/sec write SST data time = 0.99's, 195.510'MB/sec dict compress time = 0.00's, 8684146.000'MB/sec z-dict build time = 0.00's, sample length = 8.684'MB, throughput = 178778.096'MB/sec zip my value time = 8.13's, unzip length = 0.867'GB zip my value throughput = 106.717'MB/sec zip pipeline throughput = 106.717'MB/sec entries = 7375152 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.59 avg-zval = 23.22 usrkeys = 7375152 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.59 avg-zval = 23.22 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.1475 GB value = 0.8672 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0141 GB value = 0.1712 GB dict = 8.68 MB all = 0.1940 GB } UnZip/Zip{ index = 10.4527 value = 5.0652 dict = 1.00 all = 5.5334 } Zip/UnZip{ index = 0.0957 value = 0.1974 dict = 1.00 all = 0.1807 } ---------------------------- total value len = 5.203179 GB avg = 0.118 KB (by entry num) total key len = 1.239272 GB avg = 0.028 KB total ukey len = 0.885195 GB avg = 0.020 KB total ukey num = 0.044259728 Billion total entry num = 0.044259728 Billion write speed all = 2.436206489 MB/sec (with seq num) write speed all = 2.302312318 MB/sec (without seq num) 2024/07/08-17:34:55.601604 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634221: 7375152 keys, 194048761 bytes, temperature: kWarm 2024/07/08-17:34:55.601668 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431295601633, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634221, "file_size": 194048761, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 171215152, "index_size": 14111424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206504256, "raw_average_key_size": 28, "raw_value_size": 867237628, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7375152, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431275, "oldest_key_time": 0, "file_creation_time": 1720431275, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634221, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:34:55.746384 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634222.sst) /dev/shm/ToplingTemp/Topling-17573-Wq4Yt0 2024/07/08-17:35:01.661560 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634222.sst): first pass time = 5.92's, 181.524'MB/sec 2024/07/08-17:35:09.767815 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634222.sst): old prealloc_size = 1073741824, real_size = 183208800 2024/07/08-17:35:09.771797 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634222.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7382956 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8670 GB zip-val = 0.1673 GB avg-val = 117.44 avg-zval = 22.66 2024/07/08-17:35:09.771853 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634222.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:35:15.394724 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634222.sst): second pass time = 8.10's, 106.996'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.63's, 33.781'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3561.432'MB/sec (index lex order gen) rebuild zvType time = 4.63's, 0.399'MB/sec write SST data time = 1.00's, 190.473'MB/sec dict compress time = 0.00's, 8681792.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 171954.129'MB/sec zip my value time = 8.10's, unzip length = 0.867'GB zip my value throughput = 106.996'MB/sec zip pipeline throughput = 106.996'MB/sec entries = 7382956 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.44 avg-zval = 22.66 usrkeys = 7382956 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.44 avg-zval = 22.66 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.1477 GB value = 0.8670 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1673 GB dict = 8.68 MB all = 0.1901 GB } UnZip/Zip{ index = 10.5216 value = 5.1815 dict = 1.00 all = 5.6488 } Zip/UnZip{ index = 0.0950 value = 0.1930 dict = 1.00 all = 0.1770 } ---------------------------- total value len = 6.070198 GB avg = 0.118 KB (by entry num) total key len = 1.445995 GB avg = 0.028 KB total ukey len = 1.032854 GB avg = 0.020 KB total ukey num = 0.051642684 Billion total entry num = 0.051642684 Billion write speed all = 2.821094984 MB/sec (with seq num) write speed all = 2.666028277 MB/sec (without seq num) 2024/07/08-17:35:15.601023 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634222: 7382956 keys, 190082951 bytes, temperature: kWarm 2024/07/08-17:35:15.601096 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431315601057, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634222, "file_size": 190082951, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 167329040, "index_size": 14033856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206722768, "raw_average_key_size": 28, "raw_value_size": 867019064, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7382956, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431295, "oldest_key_time": 0, "file_creation_time": 1720431295, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634222, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:35:15.746233 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634223.sst) /dev/shm/ToplingTemp/Topling-17573-ih00NZ 2024/07/08-17:35:21.627242 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634223.sst): first pass time = 5.88's, 182.579'MB/sec 2024/07/08-17:35:29.758086 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634223.sst): old prealloc_size = 1073741824, real_size = 179422576 2024/07/08-17:35:29.762485 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634223.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7382902 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8670 GB zip-val = 0.1635 GB avg-val = 117.44 avg-zval = 22.15 2024/07/08-17:35:29.762542 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634223.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:35:35.323105 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634223.sst): second pass time = 8.13's, 106.665'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.57's, 33.476'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3220.660'MB/sec (index lex order gen) rebuild zvType time = 4.57's, 0.403'MB/sec write SST data time = 0.99's, 188.953'MB/sec dict compress time = 0.00's, 8681270.000'MB/sec z-dict build time = 0.00's, sample length = 8.681'MB, throughput = 170944.983'MB/sec zip my value time = 8.13's, unzip length = 0.867'GB zip my value throughput = 106.665'MB/sec zip pipeline throughput = 106.665'MB/sec entries = 7382902 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.44 avg-zval = 22.15 usrkeys = 7382902 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.44 avg-zval = 22.15 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.1477 GB value = 0.8670 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1635 GB dict = 8.68 MB all = 0.1863 GB } UnZip/Zip{ index = 10.5218 value = 5.3015 dict = 1.00 all = 5.7636 } Zip/UnZip{ index = 0.0950 value = 0.1886 dict = 1.00 all = 0.1735 } ---------------------------- total value len = 6.937219 GB avg = 0.118 KB (by entry num) total key len = 1.652716 GB avg = 0.028 KB total ukey len = 1.180512 GB avg = 0.020 KB total ukey num = 0.059025586 Billion total entry num = 0.059025586 Billion write speed all = 3.200171830 MB/sec (with seq num) write speed all = 3.024252442 MB/sec (without seq num) 2024/07/08-17:35:35.524948 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634223: 7382902 keys, 186296253 bytes, temperature: kWarm 2024/07/08-17:35:35.525017 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431335524982, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634223, "file_size": 186296253, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 163543072, "index_size": 14033600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206721256, "raw_average_key_size": 28, "raw_value_size": 867020704, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7382902, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431315, "oldest_key_time": 0, "file_creation_time": 1720431315, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634223, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:35:35.665550 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634224.sst) /dev/shm/ToplingTemp/Topling-17573-XLotTX 2024/07/08-17:35:41.529707 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634224.sst): first pass time = 5.86's, 183.104'MB/sec 2024/07/08-17:35:49.602377 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634224.sst): old prealloc_size = 1073741824, real_size = 180840176 2024/07/08-17:35:49.606700 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634224.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7374200 prefix = 4 raw-key = 0.1180 GB zip-key = 0.0141 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.8673 GB zip-val = 0.1649 GB avg-val = 117.61 avg-zval = 22.36 2024/07/08-17:35:49.606759 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634224.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:35:55.200538 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634224.sst): second pass time = 8.07's, 107.470'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.60's, 33.532'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3295.209'MB/sec (index lex order gen) rebuild zvType time = 4.60's, 0.401'MB/sec write SST data time = 1.00's, 188.098'MB/sec dict compress time = 0.00's, 8683774.000'MB/sec z-dict build time = 0.00's, sample length = 8.684'MB, throughput = 170119.973'MB/sec zip my value time = 8.07's, unzip length = 0.867'GB zip my value throughput = 107.470'MB/sec zip pipeline throughput = 107.470'MB/sec entries = 7374200 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.61 avg-zval = 22.36 usrkeys = 7374200 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.61 avg-zval = 22.36 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1475 GB value = 0.8673 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0141 GB value = 0.1649 GB dict = 8.68 MB all = 0.1877 GB } UnZip/Zip{ index = 10.4531 value = 5.2597 dict = 1.00 all = 5.7200 } Zip/UnZip{ index = 0.0957 value = 0.1901 dict = 1.00 all = 0.1748 } ---------------------------- total value len = 7.804483 GB avg = 0.118 KB (by entry num) total key len = 1.859194 GB avg = 0.028 KB total ukey len = 1.327996 GB avg = 0.020 KB total ukey num = 0.066399786 Billion total entry num = 0.066399786 Billion write speed all = 3.573728708 MB/sec (with seq num) write speed all = 3.377286034 MB/sec (without seq num) 2024/07/08-17:35:55.405489 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634224: 7374200 keys, 187718277 bytes, temperature: kWarm 2024/07/08-17:35:55.405559 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431355405523, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634224, "file_size": 187718277, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 164887328, "index_size": 14109120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206477600, "raw_average_key_size": 28, "raw_value_size": 867264296, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7374200, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431335, "oldest_key_time": 0, "file_creation_time": 1720431335, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634224, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:35:55.546495 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634225.sst) /dev/shm/ToplingTemp/Topling-17573-xOIl5W 2024/07/08-17:36:00.875322 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634225.sst): first pass time = 5.33's, 201.499'MB/sec 2024/07/08-17:36:09.327301 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634225.sst): old prealloc_size = 1073741824, real_size = 188023392 2024/07/08-17:36:09.331798 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634225.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7378381 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1722 GB avg-val = 117.53 avg-zval = 23.33 2024/07/08-17:36:09.332464 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634225.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:36:18.827011 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634225.sst): second pass time = 8.45's, 102.637'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 9.50's, 20.520'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3147.053'MB/sec (index lex order gen) rebuild zvType time = 7.13's, 0.259'MB/sec write SST data time = 2.37's, 82.280'MB/sec dict compress time = 0.00's, 8682392.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 178341.796'MB/sec zip my value time = 8.45's, unzip length = 0.867'GB zip my value throughput = 102.637'MB/sec zip pipeline throughput = 102.637'MB/sec entries = 7378381 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.53 avg-zval = 23.33 usrkeys = 7378381 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.53 avg-zval = 23.33 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1722 GB dict = 8.68 MB all = 0.1949 GB } UnZip/Zip{ index = 10.5215 value = 5.0371 dict = 1.00 all = 5.5084 } Zip/UnZip{ index = 0.0950 value = 0.1985 dict = 1.00 all = 0.1815 } ---------------------------- total value len = 8.671630 GB avg = 0.118 KB (by entry num) total key len = 2.065789 GB avg = 0.028 KB total ukey len = 1.475563 GB avg = 0.020 KB total ukey num = 0.073778167 Billion total entry num = 0.073778167 Billion write speed all = 3.936415971 MB/sec (with seq num) write speed all = 3.720035064 MB/sec (without seq num) 2024/07/08-17:36:19.066964 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634225: 7378381 keys, 194928799 bytes, temperature: kWarm 2024/07/08-17:36:19.067045 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431379067001, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634225, "file_size": 194928799, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 172153360, "index_size": 14025280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206594668, "raw_average_key_size": 28, "raw_value_size": 867147170, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7378381, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431355, "oldest_key_time": 0, "file_creation_time": 1720431355, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634225, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:36:19.306753 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634226.sst) /dev/shm/ToplingTemp/Topling-17573-c1cFzX 2024/07/08-17:36:28.699188 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634226.sst): first pass time = 9.39's, 114.321'MB/sec 2024/07/08-17:36:41.280992 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634226.sst): old prealloc_size = 1073741824, real_size = 207539232 2024/07/08-17:36:41.285082 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634226.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7365397 prefix = 4 raw-key = 0.1178 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8675 GB zip-val = 0.1917 GB avg-val = 117.78 avg-zval = 26.03 2024/07/08-17:36:41.285135 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634226.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:36:46.872369 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634226.sst): second pass time = 12.58's, 68.964'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.59's, 38.354'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3451.488'MB/sec (index lex order gen) rebuild zvType time = 4.52's, 0.407'MB/sec write SST data time = 1.06's, 201.808'MB/sec dict compress time = 0.00's, 8687092.000'MB/sec z-dict build time = 0.00's, sample length = 8.687'MB, throughput = 148426.258'MB/sec zip my value time = 12.58's, unzip length = 0.868'GB zip my value throughput = 68.964'MB/sec zip pipeline throughput = 92.805'MB/sec entries = 7365397 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.78 avg-zval = 26.03 usrkeys = 7365397 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.78 avg-zval = 26.03 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.1473 GB value = 0.8675 GB dict = 8.69 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1917 GB dict = 8.69 MB all = 0.2145 GB } UnZip/Zip{ index = 10.5215 value = 4.5254 dict = 1.00 all = 5.0069 } Zip/UnZip{ index = 0.0950 value = 0.2210 dict = 1.00 all = 0.1997 } ---------------------------- total value len = 9.539141 GB avg = 0.118 KB (by entry num) total key len = 2.272020 GB avg = 0.028 KB total ukey len = 1.622871 GB avg = 0.020 KB total ukey num = 0.081143564 Billion total entry num = 0.081143564 Billion write speed all = 4.285990559 MB/sec (with seq num) write speed all = 4.050429937 MB/sec (without seq num) 2024/07/08-17:36:47.120386 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634226: 7365397 keys, 214452411 bytes, temperature: kWarm 2024/07/08-17:36:47.120454 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431407120419, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634226, "file_size": 214452411, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 191696976, "index_size": 14000704, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206231116, "raw_average_key_size": 28, "raw_value_size": 867510786, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7365397, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431379, "oldest_key_time": 0, "file_creation_time": 1720431379, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634226, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:36:47.267887 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634227.sst) /dev/shm/ToplingTemp/Topling-17573-mST6aZ 2024/07/08-17:36:52.923145 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634227.sst): first pass time = 5.66's, 189.867'MB/sec 2024/07/08-17:37:00.923869 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634227.sst): old prealloc_size = 1073741824, real_size = 193463696 2024/07/08-17:37:00.928026 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634227.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7378544 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1776 GB avg-val = 117.52 avg-zval = 24.07 2024/07/08-17:37:00.928084 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634227.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:37:06.548481 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634227.sst): second pass time = 8.00's, 108.418'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.62's, 35.618'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3408.332'MB/sec (index lex order gen) rebuild zvType time = 4.63's, 0.398'MB/sec write SST data time = 0.99's, 202.722'MB/sec dict compress time = 0.00's, 8682362.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 150808.761'MB/sec zip my value time = 8.00's, unzip length = 0.867'GB zip my value throughput = 108.418'MB/sec zip pipeline throughput = 108.418'MB/sec entries = 7378544 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 24.07 usrkeys = 7378544 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 24.07 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1776 GB dict = 8.68 MB all = 0.2003 GB } UnZip/Zip{ index = 10.5213 value = 4.8828 dict = 1.00 all = 5.3596 } Zip/UnZip{ index = 0.0950 value = 0.2048 dict = 1.00 all = 0.1866 } ---------------------------- total value len = 10.406284 GB avg = 0.118 KB (by entry num) total key len = 2.478619 GB avg = 0.028 KB total ukey len = 1.770442 GB avg = 0.020 KB total ukey num = 0.088522108 Billion total entry num = 0.088522108 Billion write speed all = 4.642478801 MB/sec (with seq num) write speed all = 4.387320020 MB/sec (without seq num) 2024/07/08-17:37:06.763860 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634227: 7378544 keys, 200339521 bytes, temperature: kWarm 2024/07/08-17:37:06.763928 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431426763894, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634227, "file_size": 200339521, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 177592960, "index_size": 14025920, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206599232, "raw_average_key_size": 28, "raw_value_size": 867142680, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7378544, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431407, "oldest_key_time": 0, "file_creation_time": 1720431407, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634227, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:37:06.905710 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634228.sst) /dev/shm/ToplingTemp/Topling-17573-Q0EaPX 2024/07/08-17:37:12.590991 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634228.sst): first pass time = 5.69's, 188.865'MB/sec 2024/07/08-17:37:20.725570 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634228.sst): old prealloc_size = 1073741824, real_size = 194443168 2024/07/08-17:37:20.729642 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634228.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7378813 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1786 GB avg-val = 117.52 avg-zval = 24.20 2024/07/08-17:37:20.729706 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634228.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:37:26.428815 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634228.sst): second pass time = 8.13's, 106.629'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.70's, 35.299'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3476.237'MB/sec (index lex order gen) rebuild zvType time = 4.71's, 0.392'MB/sec write SST data time = 0.99's, 202.899'MB/sec dict compress time = 0.00's, 8681886.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 155494.609'MB/sec zip my value time = 8.13's, unzip length = 0.867'GB zip my value throughput = 106.629'MB/sec zip pipeline throughput = 106.629'MB/sec entries = 7378813 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 24.20 usrkeys = 7378813 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 24.20 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1786 GB dict = 8.68 MB all = 0.2013 GB } UnZip/Zip{ index = 10.5218 value = 4.8559 dict = 1.00 all = 5.3335 } Zip/UnZip{ index = 0.0950 value = 0.2059 dict = 1.00 all = 0.1875 } ---------------------------- total value len = 11.273419 GB avg = 0.118 KB (by entry num) total key len = 2.685226 GB avg = 0.028 KB total ukey len = 1.918018 GB avg = 0.020 KB total ukey num = 0.095900921 Billion total entry num = 0.095900921 Billion write speed all = 4.993583207 MB/sec (with seq num) write speed all = 4.719121471 MB/sec (without seq num) 2024/07/08-17:37:26.646390 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634228: 7378813 keys, 201318437 bytes, temperature: kWarm 2024/07/08-17:37:26.646459 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431446646425, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634228, "file_size": 201318437, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 178572496, "index_size": 14025792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206606764, "raw_average_key_size": 28, "raw_value_size": 867135082, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7378813, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431426, "oldest_key_time": 0, "file_creation_time": 1720431426, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634228, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:37:26.794242 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634229.sst) /dev/shm/ToplingTemp/Topling-17573-NM3qGY 2024/07/08-17:37:32.215181 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634229.sst): first pass time = 5.42's, 198.074'MB/sec 2024/07/08-17:37:40.205123 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634229.sst): old prealloc_size = 1073741824, real_size = 192082720 2024/07/08-17:37:40.209322 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634229.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7378678 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1762 GB avg-val = 117.52 avg-zval = 23.88 2024/07/08-17:37:40.209375 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634229.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:37:45.828701 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634229.sst): second pass time = 7.99's, 108.564'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.62's, 35.379'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3375.703'MB/sec (index lex order gen) rebuild zvType time = 4.63's, 0.399'MB/sec write SST data time = 0.99's, 200.249'MB/sec dict compress time = 0.00's, 8681994.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 183334.614'MB/sec zip my value time = 7.99's, unzip length = 0.867'GB zip my value throughput = 108.564'MB/sec zip pipeline throughput = 108.564'MB/sec entries = 7378678 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 23.88 usrkeys = 7378678 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 23.88 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 7 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1762 GB dict = 8.68 MB all = 0.1990 GB } UnZip/Zip{ index = 10.5219 value = 4.9210 dict = 1.00 all = 5.3968 } Zip/UnZip{ index = 0.0950 value = 0.2032 dict = 1.00 all = 0.1853 } ---------------------------- total value len = 12.140558 GB avg = 0.118 KB (by entry num) total key len = 2.891829 GB avg = 0.028 KB total ukey len = 2.065592 GB avg = 0.020 KB total ukey num = 0.103279599 Billion total entry num = 0.103279599 Billion write speed all = 5.340640179 MB/sec (with seq num) write speed all = 5.047098403 MB/sec (without seq num) 2024/07/08-17:37:46.047547 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634229: 7378678 keys, 198958097 bytes, temperature: kWarm 2024/07/08-17:37:46.047620 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431466047584, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634229, "file_size": 198958097, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 176212432, "index_size": 14025408, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206602984, "raw_average_key_size": 28, "raw_value_size": 867138860, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7378678, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431446, "oldest_key_time": 0, "file_creation_time": 1720431446, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634229, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:37:46.199614 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634230.sst) /dev/shm/ToplingTemp/Topling-17573-lX9ZuZ 2024/07/08-17:37:51.939012 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634230.sst): first pass time = 5.74's, 187.084'MB/sec 2024/07/08-17:38:08.147201 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634230.sst): old prealloc_size = 1073741824, real_size = 184889616 2024/07/08-17:38:08.151664 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634230.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7378718 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1690 GB avg-val = 117.52 avg-zval = 22.91 2024/07/08-17:38:08.151730 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634230.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:38:14.951424 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634230.sst): second pass time = 16.21's, 53.509'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 6.80's, 28.183'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3163.831'MB/sec (index lex order gen) rebuild zvType time = 4.79's, 0.385'MB/sec write SST data time = 2.01's, 95.617'MB/sec dict compress time = 0.00's, 8682142.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 152631.577'MB/sec zip my value time = 16.21's, unzip length = 0.867'GB zip my value throughput = 53.509'MB/sec zip pipeline throughput = 139.832'MB/sec entries = 7378718 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 22.91 usrkeys = 7378718 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 22.91 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1690 GB dict = 8.68 MB all = 0.1918 GB } UnZip/Zip{ index = 10.5214 value = 5.1304 dict = 1.00 all = 5.5993 } Zip/UnZip{ index = 0.0950 value = 0.1949 dict = 1.00 all = 0.1786 } ---------------------------- total value len = 13.007695 GB avg = 0.118 KB (by entry num) total key len = 3.098433 GB avg = 0.028 KB total ukey len = 2.213166 GB avg = 0.020 KB total ukey num = 0.110658317 Billion total entry num = 0.110658317 Billion write speed all = 5.663516345 MB/sec (with seq num) write speed all = 5.352223562 MB/sec (without seq num) 2024/07/08-17:38:15.207494 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634230: 7378718 keys, 191765157 bytes, temperature: kWarm 2024/07/08-17:38:15.207583 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431495207536, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634230, "file_size": 191765157, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 169018560, "index_size": 14026176, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206604104, "raw_average_key_size": 28, "raw_value_size": 867137724, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7378718, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431466, "oldest_key_time": 0, "file_creation_time": 1720431466, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634230, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:38:15.384546 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634231.sst) /dev/shm/ToplingTemp/Topling-17573-yJQXM0 2024/07/08-17:38:21.505411 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634231.sst): first pass time = 6.12's, 175.424'MB/sec 2024/07/08-17:38:29.628032 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634231.sst): old prealloc_size = 1073741824, real_size = 226045440 2024/07/08-17:38:29.628096 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634231.sst): index type = FixedLenKeyIndex , store type = DictZipBlobStore usrkeys = 7363663 prefix = 4 raw-key = 0.1178 GB zip-key = 0.0634 GB avg-key = 16.00 avg-zkey = 8.61 raw-val = 0.8676 GB zip-val = 0.1608 GB avg-val = 117.82 avg-zval = 21.84 2024/07/08-17:38:30.487397 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634231.sst): second pass time = 8.12's, 106.839'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 271.035'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2518832.982'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 12.392'MB/sec write SST data time = 0.71's, 327.692'MB/sec dict compress time = 0.00's, 8687354.000'MB/sec z-dict build time = 0.00's, sample length = 8.687'MB, throughput = 170587.794'MB/sec zip my value time = 8.12's, unzip length = 0.868'GB zip my value throughput = 106.839'MB/sec zip pipeline throughput = 106.839'MB/sec entries = 7363663 avg-key = 28.00 avg-zkey = 8.61 avg-val = 117.82 avg-zval = 21.84 usrkeys = 7363663 avg-key = 20.00 avg-zkey = 8.61 avg-val = 117.82 avg-zval = 21.84 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 5 UnZipSize{ index = 0.1473 GB value = 0.8676 GB dict = 8.69 MB all = 1.0737 GB } __ZipSize{ index = 0.0634 GB value = 0.1608 GB dict = 8.69 MB all = 0.2329 GB } UnZip/Zip{ index = 2.3234 value = 5.3947 dict = 1.00 all = 4.6100 } Zip/UnZip{ index = 0.4304 value = 0.1854 dict = 1.00 all = 0.2169 } ---------------------------- total value len = 13.875255 GB avg = 0.118 KB (by entry num) total key len = 3.304615 GB avg = 0.028 KB total ukey len = 2.360440 GB avg = 0.020 KB total ukey num = 0.118021980 Billion total entry num = 0.118021980 Billion write speed all = 6.008260776 MB/sec (with seq num) write speed all = 5.678057192 MB/sec (without seq num) 2024/07/08-17:38:30.737439 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634231: 7363663 keys, 232915117 bytes, temperature: kWarm 2024/07/08-17:38:30.737511 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431510737476, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634231, "file_size": 232915117, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 160817904, "index_size": 63386432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206182564, "raw_average_key_size": 28, "raw_value_size": 867559262, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7363663, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-FixedLenKeyIndex-DictZipBlobStore", "creation_time": 1720431495, "oldest_key_time": 0, "file_creation_time": 1720431495, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634231, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:38:30.887590 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634232.sst) /dev/shm/ToplingTemp/Topling-17573-CN3Zw1 2024/07/08-17:38:36.486662 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634232.sst): first pass time = 5.60's, 191.773'MB/sec 2024/07/08-17:38:44.446177 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634232.sst): old prealloc_size = 1073741824, real_size = 184201184 2024/07/08-17:38:44.450736 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634232.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7369580 prefix = 4 raw-key = 0.1179 GB zip-key = 0.0141 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.8674 GB zip-val = 0.1683 GB avg-val = 117.70 avg-zval = 22.83 2024/07/08-17:38:44.450810 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634232.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:38:50.077636 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634232.sst): second pass time = 7.96's, 109.013'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.63's, 33.931'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3123.678'MB/sec (index lex order gen) rebuild zvType time = 4.62's, 0.399'MB/sec write SST data time = 1.00's, 190.147'MB/sec dict compress time = 0.00's, 8684634.000'MB/sec z-dict build time = 0.00's, sample length = 8.685'MB, throughput = 198769.431'MB/sec zip my value time = 7.96's, unzip length = 0.867'GB zip my value throughput = 109.013'MB/sec zip pipeline throughput = 109.013'MB/sec entries = 7369580 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.70 avg-zval = 22.83 usrkeys = 7369580 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.70 avg-zval = 22.83 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.1474 GB value = 0.8674 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0141 GB value = 0.1683 GB dict = 8.68 MB all = 0.1911 GB } UnZip/Zip{ index = 10.4529 value = 5.1551 dict = 1.00 all = 5.6193 } Zip/UnZip{ index = 0.0957 value = 0.1940 dict = 1.00 all = 0.1780 } ---------------------------- total value len = 14.742648 GB avg = 0.118 KB (by entry num) total key len = 3.510964 GB avg = 0.028 KB total ukey len = 2.507831 GB avg = 0.020 KB total ukey num = 0.125391560 Billion total entry num = 0.125391560 Billion write speed all = 6.340337981 MB/sec (with seq num) write speed all = 5.991902891 MB/sec (without seq num) 2024/07/08-17:38:50.285139 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634232: 7369580 keys, 191081441 bytes, temperature: kWarm 2024/07/08-17:38:50.285209 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431530285172, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634232, "file_size": 191081441, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 168258064, "index_size": 14100544, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206348240, "raw_average_key_size": 28, "raw_value_size": 867393728, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7369580, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431510, "oldest_key_time": 0, "file_creation_time": 1720431510, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634232, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:38:50.438634 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634233.sst) /dev/shm/ToplingTemp/Topling-17573-HVidC1 2024/07/08-17:38:56.017694 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634233.sst): first pass time = 5.58's, 192.461'MB/sec 2024/07/08-17:39:04.077082 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634233.sst): old prealloc_size = 1073741824, real_size = 190906208 2024/07/08-17:39:04.081336 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634233.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7378566 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1750 GB avg-val = 117.52 avg-zval = 23.72 2024/07/08-17:39:04.081392 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634233.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:39:09.690365 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634233.sst): second pass time = 8.06's, 107.629'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.61's, 35.235'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3331.001'MB/sec (index lex order gen) rebuild zvType time = 4.60's, 0.401'MB/sec write SST data time = 1.01's, 196.141'MB/sec dict compress time = 0.00's, 8682306.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 172926.745'MB/sec zip my value time = 8.06's, unzip length = 0.867'GB zip my value throughput = 107.629'MB/sec zip pipeline throughput = 107.629'MB/sec entries = 7378566 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 23.72 usrkeys = 7378566 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 23.72 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1750 GB dict = 8.68 MB all = 0.1978 GB } UnZip/Zip{ index = 10.5220 value = 4.9541 dict = 1.00 all = 5.4289 } Zip/UnZip{ index = 0.0950 value = 0.2019 dict = 1.00 all = 0.1842 } ---------------------------- total value len = 15.609790 GB avg = 0.118 KB (by entry num) total key len = 3.717564 GB avg = 0.028 KB total ukey len = 2.655403 GB avg = 0.020 KB total ukey num = 0.132770126 Billion total entry num = 0.132770126 Billion write speed all = 6.667874649 MB/sec (with seq num) write speed all = 6.301432533 MB/sec (without seq num) 2024/07/08-17:39:09.911799 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634233: 7378566 keys, 197781961 bytes, temperature: kWarm 2024/07/08-17:39:09.911869 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431549911834, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634233, "file_size": 197781961, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 175036304, "index_size": 14025088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206599848, "raw_average_key_size": 28, "raw_value_size": 867142016, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7378566, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431530, "oldest_key_time": 0, "file_creation_time": 1720431530, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634233, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:39:10.061526 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634234.sst) /dev/shm/ToplingTemp/Topling-17573-VqKTlX 2024/07/08-17:39:15.665210 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634234.sst): first pass time = 5.60's, 191.615'MB/sec 2024/07/08-17:39:23.777959 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634234.sst): old prealloc_size = 1073741824, real_size = 194239792 2024/07/08-17:39:23.782247 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634234.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7378636 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1784 GB avg-val = 117.52 avg-zval = 24.17 2024/07/08-17:39:23.782300 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634234.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:39:29.474122 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634234.sst): second pass time = 8.11's, 106.920'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.70's, 35.307'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3308.257'MB/sec (index lex order gen) rebuild zvType time = 4.69's, 0.393'MB/sec write SST data time = 1.00's, 201.036'MB/sec dict compress time = 0.00's, 8682402.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 182510.763'MB/sec zip my value time = 8.11's, unzip length = 0.867'GB zip my value throughput = 106.920'MB/sec zip pipeline throughput = 106.920'MB/sec entries = 7378636 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 24.17 usrkeys = 7378636 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.52 avg-zval = 24.17 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1784 GB dict = 8.68 MB all = 0.2011 GB } UnZip/Zip{ index = 10.5219 value = 4.8615 dict = 1.00 all = 5.3389 } Zip/UnZip{ index = 0.0950 value = 0.2057 dict = 1.00 all = 0.1873 } ---------------------------- total value len = 16.476930 GB avg = 0.118 KB (by entry num) total key len = 3.924165 GB avg = 0.028 KB total ukey len = 2.802975 GB avg = 0.020 KB total ukey num = 0.140148762 Billion total entry num = 0.140148762 Billion write speed all = 6.990598959 MB/sec (with seq num) write speed all = 6.606414187 MB/sec (without seq num) 2024/07/08-17:39:29.691450 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634234: 7378636 keys, 201115689 bytes, temperature: kWarm 2024/07/08-17:39:29.691521 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431569691485, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634234, "file_size": 201115689, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 178369696, "index_size": 14025280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206601808, "raw_average_key_size": 28, "raw_value_size": 867140036, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7378636, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431550, "oldest_key_time": 0, "file_creation_time": 1720431550, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634234, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:39:29.840964 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634235.sst) /dev/shm/ToplingTemp/Topling-17573-SdSAOZ 2024/07/08-17:39:35.088638 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634235.sst): first pass time = 5.25's, 204.615'MB/sec 2024/07/08-17:39:43.269792 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634235.sst): old prealloc_size = 1073741824, real_size = 193777888 2024/07/08-17:39:43.274000 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634235.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7378969 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1779 GB avg-val = 117.51 avg-zval = 24.11 2024/07/08-17:39:43.274051 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634235.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:39:53.664377 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634235.sst): second pass time = 8.18's, 106.022'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 10.39's, 19.304'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3361.764'MB/sec (index lex order gen) rebuild zvType time = 8.57's, 0.215'MB/sec write SST data time = 1.82's, 110.275'MB/sec dict compress time = 0.00's, 8681964.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 185849.599'MB/sec zip my value time = 8.18's, unzip length = 0.867'GB zip my value throughput = 106.022'MB/sec zip pipeline throughput = 106.022'MB/sec entries = 7378969 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.51 avg-zval = 24.11 usrkeys = 7378969 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.51 avg-zval = 24.11 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1779 GB dict = 8.68 MB all = 0.2007 GB } UnZip/Zip{ index = 10.5219 value = 4.8741 dict = 1.00 all = 5.3512 } Zip/UnZip{ index = 0.0950 value = 0.2052 dict = 1.00 all = 0.1869 } ---------------------------- total value len = 17.344061 GB avg = 0.118 KB (by entry num) total key len = 4.130776 GB avg = 0.028 KB total ukey len = 2.950555 GB avg = 0.020 KB total ukey num = 0.147527731 Billion total entry num = 0.147527731 Billion write speed all = 7.298031999 MB/sec (with seq num) write speed all = 6.896944134 MB/sec (without seq num) 2024/07/08-17:39:53.935621 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634235: 7378969 keys, 200653171 bytes, temperature: kWarm 2024/07/08-17:39:53.935693 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431593935651, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634235, "file_size": 200653171, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 177906960, "index_size": 14025984, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206611132, "raw_average_key_size": 28, "raw_value_size": 867130822, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7378969, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431569, "oldest_key_time": 0, "file_creation_time": 1720431569, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634235, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:39:54.129601 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634236.sst) /dev/shm/ToplingTemp/Topling-17573-zLGMU0 2024/07/08-17:40:00.577425 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634236.sst): first pass time = 6.45's, 166.529'MB/sec 2024/07/08-17:40:09.085466 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634236.sst): old prealloc_size = 1073741824, real_size = 190446880 2024/07/08-17:40:09.089701 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634236.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7378365 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1746 GB avg-val = 117.53 avg-zval = 23.66 2024/07/08-17:40:09.089758 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634236.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:40:14.716914 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634236.sst): second pass time = 8.51's, 101.956'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.63's, 35.039'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3344.847'MB/sec (index lex order gen) rebuild zvType time = 4.62's, 0.399'MB/sec write SST data time = 1.01's, 196.216'MB/sec dict compress time = 0.00's, 8682428.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 190387.422'MB/sec zip my value time = 8.51's, unzip length = 0.867'GB zip my value throughput = 101.956'MB/sec zip pipeline throughput = 101.956'MB/sec entries = 7378365 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.53 avg-zval = 23.66 usrkeys = 7378365 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.53 avg-zval = 23.66 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1746 GB dict = 8.68 MB all = 0.1973 GB } UnZip/Zip{ index = 10.5215 value = 4.9671 dict = 1.00 all = 5.4415 } Zip/UnZip{ index = 0.0950 value = 0.2013 dict = 1.00 all = 0.1838 } ---------------------------- total value len = 18.211209 GB avg = 0.118 KB (by entry num) total key len = 4.337371 GB avg = 0.028 KB total ukey len = 3.098122 GB avg = 0.020 KB total ukey num = 0.154906096 Billion total entry num = 0.154906096 Billion write speed all = 7.608498429 MB/sec (with seq num) write speed all = 7.190342498 MB/sec (without seq num) 2024/07/08-17:40:14.931999 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634236: 7378365 keys, 197322819 bytes, temperature: kWarm 2024/07/08-17:40:14.932068 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431614932033, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634236, "file_size": 197322819, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 174576784, "index_size": 14025344, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206594220, "raw_average_key_size": 28, "raw_value_size": 867147666, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7378365, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431594, "oldest_key_time": 0, "file_creation_time": 1720431594, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634236, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:40:15.078270 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634237.sst) /dev/shm/ToplingTemp/Topling-17573-DbnbdY 2024/07/08-17:40:21.032790 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634237.sst): first pass time = 5.95's, 180.326'MB/sec 2024/07/08-17:40:29.289549 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634237.sst): old prealloc_size = 1073741824, real_size = 182174656 2024/07/08-17:40:29.293683 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634237.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7382188 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8670 GB zip-val = 0.1663 GB avg-val = 117.45 avg-zval = 22.53 2024/07/08-17:40:29.293741 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634237.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:40:34.856030 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634237.sst): second pass time = 8.25's, 105.039'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.57's, 33.967'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3426.702'MB/sec (index lex order gen) rebuild zvType time = 4.56's, 0.405'MB/sec write SST data time = 1.01's, 188.081'MB/sec dict compress time = 0.00's, 8682126.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 140621.726'MB/sec zip my value time = 8.25's, unzip length = 0.867'GB zip my value throughput = 105.039'MB/sec zip pipeline throughput = 105.039'MB/sec entries = 7382188 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.45 avg-zval = 22.53 usrkeys = 7382188 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.45 avg-zval = 22.53 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1476 GB value = 0.8670 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1663 GB dict = 8.68 MB all = 0.1891 GB } UnZip/Zip{ index = 10.5228 value = 5.2138 dict = 1.00 all = 5.6788 } Zip/UnZip{ index = 0.0950 value = 0.1918 dict = 1.00 all = 0.1761 } ---------------------------- total value len = 19.078249 GB avg = 0.118 KB (by entry num) total key len = 4.544072 GB avg = 0.028 KB total ukey len = 3.245766 GB avg = 0.020 KB total ukey num = 0.162288284 Billion total entry num = 0.162288284 Billion write speed all = 7.917007996 MB/sec (with seq num) write speed all = 7.481881366 MB/sec (without seq num) 2024/07/08-17:40:35.080414 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634237: 7382188 keys, 189078933 bytes, temperature: kWarm 2024/07/08-17:40:35.080484 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431635080449, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634237, "file_size": 189078933, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 166298160, "index_size": 14030784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206701264, "raw_average_key_size": 28, "raw_value_size": 867040584, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7382188, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431615, "oldest_key_time": 0, "file_creation_time": 1720431615, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634237, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:40:35.219965 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634238.sst) /dev/shm/ToplingTemp/Topling-17573-AKPL8X 2024/07/08-17:40:40.837567 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634238.sst): first pass time = 5.62's, 191.140'MB/sec 2024/07/08-17:40:49.817286 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634238.sst): old prealloc_size = 1073741824, real_size = 221497264 2024/07/08-17:40:49.821839 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634238.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7373487 prefix = 4 raw-key = 0.1180 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8673 GB zip-val = 0.2056 GB avg-val = 117.62 avg-zval = 27.89 2024/07/08-17:40:49.821894 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634238.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:40:55.328455 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634238.sst): second pass time = 8.98's, 96.615'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.51's, 41.444'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3107.690'MB/sec (index lex order gen) rebuild zvType time = 4.50's, 0.409'MB/sec write SST data time = 1.00's, 227.855'MB/sec dict compress time = 0.00's, 8684654.000'MB/sec z-dict build time = 0.00's, sample length = 8.685'MB, throughput = 180340.428'MB/sec zip my value time = 8.98's, unzip length = 0.867'GB zip my value throughput = 96.615'MB/sec zip pipeline throughput = 96.615'MB/sec entries = 7373487 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.62 avg-zval = 27.89 usrkeys = 7373487 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.62 avg-zval = 27.89 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.1475 GB value = 0.8673 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.2056 GB dict = 8.68 MB all = 0.2284 GB } UnZip/Zip{ index = 10.5210 value = 4.2175 dict = 1.00 all = 4.7010 } Zip/UnZip{ index = 0.0950 value = 0.2371 dict = 1.00 all = 0.2127 } ---------------------------- total value len = 19.945534 GB avg = 0.118 KB (by entry num) total key len = 4.750530 GB avg = 0.028 KB total ukey len = 3.393235 GB avg = 0.020 KB total ukey num = 0.169661771 Billion total entry num = 0.169661771 Billion write speed all = 8.220468712 MB/sec (with seq num) write speed all = 7.768672242 MB/sec (without seq num) 2024/07/08-17:40:55.562602 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634238: 7373487 keys, 228406005 bytes, temperature: kWarm 2024/07/08-17:40:55.562678 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431655562637, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634238, "file_size": 228406005, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 205637088, "index_size": 14016640, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206457636, "raw_average_key_size": 28, "raw_value_size": 867284250, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7373487, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431635, "oldest_key_time": 0, "file_creation_time": 1720431635, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634238, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:40:55.716250 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634239.sst) /dev/shm/ToplingTemp/Topling-17573-eRwIa0 2024/07/08-17:41:01.059444 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634239.sst): first pass time = 5.34's, 200.957'MB/sec 2024/07/08-17:41:09.244688 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634239.sst): old prealloc_size = 1073741824, real_size = 193601856 2024/07/08-17:41:09.249075 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634239.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7381139 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1777 GB avg-val = 117.47 avg-zval = 24.08 2024/07/08-17:41:09.249130 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634239.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:41:14.938064 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634239.sst): second pass time = 8.18's, 105.961'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.69's, 35.212'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3232.133'MB/sec (index lex order gen) rebuild zvType time = 4.68's, 0.394'MB/sec write SST data time = 1.00's, 199.494'MB/sec dict compress time = 0.00's, 8681928.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 151200.418'MB/sec zip my value time = 8.18's, unzip length = 0.867'GB zip my value throughput = 105.961'MB/sec zip pipeline throughput = 105.961'MB/sec entries = 7381139 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.47 avg-zval = 24.08 usrkeys = 7381139 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.47 avg-zval = 24.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1777 GB dict = 8.68 MB all = 0.2005 GB } UnZip/Zip{ index = 10.5209 value = 4.8787 dict = 1.00 all = 5.3559 } Zip/UnZip{ index = 0.0950 value = 0.2050 dict = 1.00 all = 0.1867 } ---------------------------- total value len = 20.812604 GB avg = 0.118 KB (by entry num) total key len = 4.957201 GB avg = 0.028 KB total ukey len = 3.540858 GB avg = 0.020 KB total ukey num = 0.177042910 Billion total entry num = 0.177042910 Billion write speed all = 8.522252581 MB/sec (with seq num) write speed all = 8.053858077 MB/sec (without seq num) 2024/07/08-17:41:15.158515 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634239: 7381139 keys, 200476623 bytes, temperature: kWarm 2024/07/08-17:41:15.158583 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431675158549, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634239, "file_size": 200476623, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 177725040, "index_size": 14031360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206671892, "raw_average_key_size": 28, "raw_value_size": 867070070, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7381139, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431655, "oldest_key_time": 0, "file_creation_time": 1720431655, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634239, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:41:15.308939 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634240.sst) /dev/shm/ToplingTemp/Topling-17573-yPlstZ 2024/07/08-17:41:21.000105 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634240.sst): first pass time = 5.69's, 188.669'MB/sec 2024/07/08-17:41:29.809973 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634240.sst): old prealloc_size = 1073741824, real_size = 185606320 2024/07/08-17:41:29.814415 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634240.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7380867 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0141 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.8671 GB zip-val = 0.1696 GB avg-val = 117.48 avg-zval = 22.98 2024/07/08-17:41:29.814474 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634240.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:41:38.497378 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634240.sst): second pass time = 8.81's, 98.451'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 8.69's, 22.156'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3197.970'MB/sec (index lex order gen) rebuild zvType time = 7.41's, 0.249'MB/sec write SST data time = 1.27's, 151.254'MB/sec dict compress time = 0.00's, 8682174.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 183683.627'MB/sec zip my value time = 8.81's, unzip length = 0.867'GB zip my value throughput = 98.451'MB/sec zip pipeline throughput = 98.451'MB/sec entries = 7380867 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.48 avg-zval = 22.98 usrkeys = 7380867 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.48 avg-zval = 22.98 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0141 GB value = 0.1696 GB dict = 8.68 MB all = 0.1925 GB } UnZip/Zip{ index = 10.4532 value = 5.1113 dict = 1.00 all = 5.5784 } Zip/UnZip{ index = 0.0957 value = 0.1956 dict = 1.00 all = 0.1793 } ---------------------------- total value len = 21.679681 GB avg = 0.118 KB (by entry num) total key len = 5.163866 GB avg = 0.028 KB total ukey len = 3.688476 GB avg = 0.020 KB total ukey num = 0.184423777 Billion total entry num = 0.184423777 Billion write speed all = 8.808715701 MB/sec (with seq num) write speed all = 8.324566089 MB/sec (without seq num) 2024/07/08-17:41:38.754747 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634240: 7380867 keys, 192481477 bytes, temperature: kWarm 2024/07/08-17:41:38.755628 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431698754790, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634240, "file_size": 192481477, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 169639136, "index_size": 14121792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206664276, "raw_average_key_size": 28, "raw_value_size": 867077614, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7380867, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431675, "oldest_key_time": 0, "file_creation_time": 1720431675, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634240, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:41:38.912035 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634241.sst) /dev/shm/ToplingTemp/Topling-17573-9ppXZY 2024/07/08-17:41:45.100565 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634241.sst): first pass time = 6.19's, 173.506'MB/sec 2024/07/08-17:41:54.235387 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634241.sst): old prealloc_size = 1073741824, real_size = 226262656 2024/07/08-17:41:54.235464 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634241.sst): index type = FixedLenKeyIndex , store type = DictZipBlobStore usrkeys = 7354397 prefix = 4 raw-key = 0.1177 GB zip-key = 0.0633 GB avg-key = 16.00 avg-zkey = 8.61 raw-val = 0.8678 GB zip-val = 0.1611 GB avg-val = 118.00 avg-zval = 21.91 2024/07/08-17:41:55.111513 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634241.sst): second pass time = 9.13's, 95.027'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 266.103'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2403179.226'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 12.187'MB/sec write SST data time = 0.73's, 321.466'MB/sec dict compress time = 0.00's, 8690818.000'MB/sec z-dict build time = 0.00's, sample length = 8.691'MB, throughput = 156820.188'MB/sec zip my value time = 9.13's, unzip length = 0.868'GB zip my value throughput = 95.027'MB/sec zip pipeline throughput = 98.236'MB/sec entries = 7354397 avg-key = 28.00 avg-zkey = 8.61 avg-val = 118.00 avg-zval = 21.91 usrkeys = 7354397 avg-key = 20.00 avg-zkey = 8.61 avg-val = 118.00 avg-zval = 21.91 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 5 UnZipSize{ index = 0.1471 GB value = 0.8678 GB dict = 8.69 MB all = 1.0737 GB } __ZipSize{ index = 0.0633 GB value = 0.1611 GB dict = 8.69 MB all = 0.2331 GB } UnZip/Zip{ index = 2.3236 value = 5.3861 dict = 1.00 all = 4.6056 } Zip/UnZip{ index = 0.4304 value = 0.1857 dict = 1.00 all = 0.2171 } ---------------------------- total value len = 22.547500 GB avg = 0.118 KB (by entry num) total key len = 5.369789 GB avg = 0.028 KB total ukey len = 3.835563 GB avg = 0.020 KB total ukey num = 0.191778174 Billion total entry num = 0.191778174 Billion write speed all = 9.111389700 MB/sec (with seq num) write speed all = 8.610663238 MB/sec (without seq num) 2024/07/08-17:41:55.361042 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634241: 7354397 keys, 233138117 bytes, temperature: kWarm 2024/07/08-17:41:55.361110 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431715361076, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634241, "file_size": 233138117, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 161121712, "index_size": 63302144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 205923116, "raw_average_key_size": 28, "raw_value_size": 867818846, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 7354397, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-FixedLenKeyIndex-DictZipBlobStore", "creation_time": 1720431698, "oldest_key_time": 0, "file_creation_time": 1720431698, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634241, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:41:55.517907 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634242.sst) /dev/shm/ToplingTemp/Topling-17573-l4rYG0 2024/07/08-17:42:01.124500 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634242.sst): first pass time = 5.61's, 191.516'MB/sec 2024/07/08-17:42:09.315419 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634242.sst): old prealloc_size = 1073741824, real_size = 187051856 2024/07/08-17:42:09.319696 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634242.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7381063 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0141 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.8671 GB zip-val = 0.1711 GB avg-val = 117.47 avg-zval = 23.18 2024/07/08-17:42:09.319754 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634242.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:42:14.983956 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634242.sst): second pass time = 8.19's, 105.890'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.67's, 34.211'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3336.227'MB/sec (index lex order gen) rebuild zvType time = 4.64's, 0.398'MB/sec write SST data time = 1.02's, 189.578'MB/sec dict compress time = 0.00's, 8682982.000'MB/sec z-dict build time = 0.00's, sample length = 8.683'MB, throughput = 134797.516'MB/sec zip my value time = 8.19's, unzip length = 0.867'GB zip my value throughput = 105.890'MB/sec zip pipeline throughput = 105.890'MB/sec entries = 7381063 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.47 avg-zval = 23.18 usrkeys = 7381063 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.47 avg-zval = 23.18 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0141 GB value = 0.1711 GB dict = 8.68 MB all = 0.1939 GB } UnZip/Zip{ index = 10.4528 value = 5.0681 dict = 1.00 all = 5.5368 } Zip/UnZip{ index = 0.0957 value = 0.1973 dict = 1.00 all = 0.1806 } ---------------------------- total value len = 23.414572 GB avg = 0.118 KB (by entry num) total key len = 5.576459 GB avg = 0.028 KB total ukey len = 3.983185 GB avg = 0.020 KB total ukey num = 0.199159237 Billion total entry num = 0.199159237 Billion write speed all = 9.400855847 MB/sec (with seq num) write speed all = 8.884208536 MB/sec (without seq num) 2024/07/08-17:42:15.194976 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634242: 7381063 keys, 193927661 bytes, temperature: kWarm 2024/07/08-17:42:15.195048 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431735195012, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634242, "file_size": 193927661, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 171083712, "index_size": 14122688, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206669764, "raw_average_key_size": 28, "raw_value_size": 867072102, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7381063, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431715, "oldest_key_time": 0, "file_creation_time": 1720431715, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634242, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:42:15.344227 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634243.sst) /dev/shm/ToplingTemp/Topling-17573-9pJKn1 2024/07/08-17:42:20.892866 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634243.sst): first pass time = 5.55's, 193.516'MB/sec 2024/07/08-17:42:29.027895 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634243.sst): old prealloc_size = 1073741824, real_size = 193385776 2024/07/08-17:42:29.032195 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634243.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7380464 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0141 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.8671 GB zip-val = 0.1774 GB avg-val = 117.48 avg-zval = 24.04 2024/07/08-17:42:29.032256 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634243.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:42:34.725611 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634243.sst): second pass time = 8.13's, 106.621'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.70's, 35.148'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3324.990'MB/sec (index lex order gen) rebuild zvType time = 4.69's, 0.393'MB/sec write SST data time = 1.00's, 199.484'MB/sec dict compress time = 0.00's, 8681992.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 175411.496'MB/sec zip my value time = 8.13's, unzip length = 0.867'GB zip my value throughput = 106.621'MB/sec zip pipeline throughput = 106.621'MB/sec entries = 7380464 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.48 avg-zval = 24.04 usrkeys = 7380464 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.48 avg-zval = 24.04 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0141 GB value = 0.1774 GB dict = 8.68 MB all = 0.2003 GB } UnZip/Zip{ index = 10.4525 value = 4.8872 dict = 1.00 all = 5.3617 } Zip/UnZip{ index = 0.0957 value = 0.2046 dict = 1.00 all = 0.1865 } ---------------------------- total value len = 24.281661 GB avg = 0.118 KB (by entry num) total key len = 5.783112 GB avg = 0.028 KB total ukey len = 4.130794 GB avg = 0.020 KB total ukey num = 0.206539701 Billion total entry num = 0.206539701 Billion write speed all = 9.687023373 MB/sec (with seq num) write speed all = 9.154638200 MB/sec (without seq num) 2024/07/08-17:42:34.944815 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634243: 7380464 keys, 200260623 bytes, temperature: kWarm 2024/07/08-17:42:34.944885 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431754944851, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634243, "file_size": 200260623, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 177418592, "index_size": 14121856, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206652992, "raw_average_key_size": 28, "raw_value_size": 867088832, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7380464, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431735, "oldest_key_time": 0, "file_creation_time": 1720431735, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634243, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:42:35.101508 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634244.sst) /dev/shm/ToplingTemp/Topling-17573-Ff6sCX 2024/07/08-17:42:40.318747 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634244.sst): first pass time = 5.22's, 205.808'MB/sec 2024/07/08-17:42:48.403314 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634244.sst): old prealloc_size = 1073741824, real_size = 191623552 2024/07/08-17:42:48.407861 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634244.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7380692 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1757 GB avg-val = 117.48 avg-zval = 23.81 2024/07/08-17:42:48.407927 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634244.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:42:54.047792 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634244.sst): second pass time = 8.08's, 107.284'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.64's, 35.167'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3112.909'MB/sec (index lex order gen) rebuild zvType time = 4.62's, 0.399'MB/sec write SST data time = 1.02's, 195.211'MB/sec dict compress time = 0.00's, 8681348.000'MB/sec z-dict build time = 0.00's, sample length = 8.681'MB, throughput = 130155.142'MB/sec zip my value time = 8.08's, unzip length = 0.867'GB zip my value throughput = 107.284'MB/sec zip pipeline throughput = 107.284'MB/sec entries = 7380692 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.48 avg-zval = 23.81 usrkeys = 7380692 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.48 avg-zval = 23.81 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1757 GB dict = 8.68 MB all = 0.1985 GB } UnZip/Zip{ index = 10.5217 value = 4.9336 dict = 1.00 all = 5.4093 } Zip/UnZip{ index = 0.0950 value = 0.2027 dict = 1.00 all = 0.1849 } ---------------------------- total value len = 25.148744 GB avg = 0.118 KB (by entry num) total key len = 5.989771 GB avg = 0.028 KB total ukey len = 4.278408 GB avg = 0.020 KB total ukey num = 0.213920393 Billion total entry num = 0.213920393 Billion write speed all = 9.970912568 MB/sec (with seq num) write speed all = 9.422914295 MB/sec (without seq num) 2024/07/08-17:42:54.267099 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634244: 7380692 keys, 198497867 bytes, temperature: kWarm 2024/07/08-17:42:54.267167 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431774267132, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634244, "file_size": 198497867, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 175748720, "index_size": 14029504, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206659376, "raw_average_key_size": 28, "raw_value_size": 867082456, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7380692, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431755, "oldest_key_time": 0, "file_creation_time": 1720431755, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634244, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:42:54.420445 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634245.sst) /dev/shm/ToplingTemp/Topling-17573-quYE10 2024/07/08-17:43:00.024321 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634245.sst): first pass time = 5.60's, 191.608'MB/sec 2024/07/08-17:43:08.129522 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634245.sst): old prealloc_size = 1073741824, real_size = 175122992 2024/07/08-17:43:08.133736 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634245.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7380867 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1592 GB avg-val = 117.48 avg-zval = 21.58 2024/07/08-17:43:08.133804 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634245.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:43:13.691068 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634245.sst): second pass time = 8.10's, 107.016'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.56's, 32.724'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3360.097'MB/sec (index lex order gen) rebuild zvType time = 4.55's, 0.405'MB/sec write SST data time = 1.00's, 181.204'MB/sec dict compress time = 0.00's, 8681314.000'MB/sec z-dict build time = 0.00's, sample length = 8.681'MB, throughput = 183281.552'MB/sec zip my value time = 8.10's, unzip length = 0.867'GB zip my value throughput = 107.016'MB/sec zip pipeline throughput = 107.016'MB/sec entries = 7380867 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.48 avg-zval = 21.58 usrkeys = 7380867 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.48 avg-zval = 21.58 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1592 GB dict = 8.68 MB all = 0.1820 GB } UnZip/Zip{ index = 10.5217 value = 5.4448 dict = 1.00 all = 5.8998 } Zip/UnZip{ index = 0.0950 value = 0.1837 dict = 1.00 all = 0.1695 } ---------------------------- total value len = 26.015821 GB avg = 0.118 KB (by entry num) total key len = 6.196435 GB avg = 0.028 KB total ukey len = 4.426025 GB avg = 0.020 KB total ukey num = 0.221301260 Billion total entry num = 0.221301260 Billion write speed all = 10.250262905 MB/sec (with seq num) write speed all = 9.686900674 MB/sec (without seq num) 2024/07/08-17:43:13.880570 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634245: 7380867 keys, 181997225 bytes, temperature: kWarm 2024/07/08-17:43:13.880639 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431793880603, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634245, "file_size": 181997225, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 159247840, "index_size": 14029760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206664276, "raw_average_key_size": 28, "raw_value_size": 867077678, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7380867, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431774, "oldest_key_time": 0, "file_creation_time": 1720431774, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634245, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:43:14.023230 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634246.sst) /dev/shm/ToplingTemp/Topling-17573-3UneG0 2024/07/08-17:43:19.634590 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634246.sst): first pass time = 5.61's, 191.353'MB/sec 2024/07/08-17:43:27.515932 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634246.sst): old prealloc_size = 1073741824, real_size = 225974896 2024/07/08-17:43:27.516007 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634246.sst): index type = FixedLenKeyIndex , store type = DictZipBlobStore usrkeys = 7354397 prefix = 4 raw-key = 0.1177 GB zip-key = 0.0633 GB avg-key = 16.00 avg-zkey = 8.61 raw-val = 0.8678 GB zip-val = 0.1608 GB avg-val = 118.00 avg-zval = 21.87 2024/07/08-17:43:28.387358 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634246.sst): second pass time = 7.88's, 110.144'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 267.208'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2375611.213'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 12.271'MB/sec write SST data time = 0.72's, 322.703'MB/sec dict compress time = 0.00's, 8690818.000'MB/sec z-dict build time = 0.00's, sample length = 8.691'MB, throughput = 191250.781'MB/sec zip my value time = 7.88's, unzip length = 0.868'GB zip my value throughput = 110.144'MB/sec zip pipeline throughput = 110.144'MB/sec entries = 7354397 avg-key = 28.00 avg-zkey = 8.61 avg-val = 118.00 avg-zval = 21.87 usrkeys = 7354397 avg-key = 20.00 avg-zkey = 8.61 avg-val = 118.00 avg-zval = 21.87 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 5 UnZipSize{ index = 0.1471 GB value = 0.8678 GB dict = 8.69 MB all = 1.0737 GB } __ZipSize{ index = 0.0633 GB value = 0.1608 GB dict = 8.69 MB all = 0.2329 GB } UnZip/Zip{ index = 2.3236 value = 5.3958 dict = 1.00 all = 4.6113 } Zip/UnZip{ index = 0.4304 value = 0.1853 dict = 1.00 all = 0.2169 } ---------------------------- total value len = 26.883640 GB avg = 0.118 KB (by entry num) total key len = 6.402358 GB avg = 0.028 KB total ukey len = 4.573113 GB avg = 0.020 KB total ukey num = 0.228655657 Billion total entry num = 0.228655657 Billion write speed all = 10.542635650 MB/sec (with seq num) write speed all = 9.963260926 MB/sec (without seq num) 2024/07/08-17:43:28.642233 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634246: 7354397 keys, 232850373 bytes, temperature: kWarm 2024/07/08-17:43:28.642303 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431808642267, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634246, "file_size": 232850373, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 160833184, "index_size": 63302912, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 205923116, "raw_average_key_size": 28, "raw_value_size": 867818846, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 7354397, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-FixedLenKeyIndex-DictZipBlobStore", "creation_time": 1720431794, "oldest_key_time": 0, "file_creation_time": 1720431794, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634246, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:43:28.796677 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634247.sst) /dev/shm/ToplingTemp/Topling-17573-X8EluY 2024/07/08-17:43:34.414651 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634247.sst): first pass time = 5.62's, 191.128'MB/sec 2024/07/08-17:43:42.698617 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634247.sst): old prealloc_size = 1073741824, real_size = 191190640 2024/07/08-17:43:42.702935 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634247.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7380794 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1753 GB avg-val = 117.48 avg-zval = 23.75 2024/07/08-17:43:42.702989 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634247.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:43:48.307550 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634247.sst): second pass time = 8.28's, 104.708'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.61's, 35.313'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3281.553'MB/sec (index lex order gen) rebuild zvType time = 4.60's, 0.401'MB/sec write SST data time = 1.01's, 196.328'MB/sec dict compress time = 0.00's, 8681836.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 181302.176'MB/sec zip my value time = 8.28's, unzip length = 0.867'GB zip my value throughput = 104.708'MB/sec zip pipeline throughput = 104.708'MB/sec entries = 7380794 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.48 avg-zval = 23.75 usrkeys = 7380794 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.48 avg-zval = 23.75 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 9 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1753 GB dict = 8.68 MB all = 0.1981 GB } UnZip/Zip{ index = 10.5209 value = 4.9459 dict = 1.00 all = 5.4211 } Zip/UnZip{ index = 0.0950 value = 0.2022 dict = 1.00 all = 0.1845 } ---------------------------- total value len = 27.750720 GB avg = 0.118 KB (by entry num) total key len = 6.609021 GB avg = 0.028 KB total ukey len = 4.720729 GB avg = 0.020 KB total ukey num = 0.236036451 Billion total entry num = 0.236036451 Billion write speed all = 10.814488856 MB/sec (with seq num) write speed all = 10.220162233 MB/sec (without seq num) 2024/07/08-17:43:48.539236 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634247: 7380794 keys, 198065395 bytes, temperature: kWarm 2024/07/08-17:43:48.539306 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431828539271, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634247, "file_size": 198065395, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 175314464, "index_size": 14030784, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206662232, "raw_average_key_size": 28, "raw_value_size": 867079716, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7380794, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431808, "oldest_key_time": 0, "file_creation_time": 1720431808, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634247, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:43:48.685072 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634248.sst) /dev/shm/ToplingTemp/Topling-17573-MLXxi1 2024/07/08-17:43:54.345925 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634248.sst): first pass time = 5.66's, 189.680'MB/sec 2024/07/08-17:44:02.476728 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634248.sst): old prealloc_size = 1073741824, real_size = 194004432 2024/07/08-17:44:02.480858 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634248.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7380822 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0141 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.8671 GB zip-val = 0.1780 GB avg-val = 117.48 avg-zval = 24.12 2024/07/08-17:44:02.480912 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634248.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:44:08.189007 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634248.sst): second pass time = 8.13's, 106.677'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.71's, 35.166'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3465.257'MB/sec (index lex order gen) rebuild zvType time = 4.70's, 0.393'MB/sec write SST data time = 1.01's, 198.424'MB/sec dict compress time = 0.00's, 8681612.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 188554.438'MB/sec zip my value time = 8.13's, unzip length = 0.867'GB zip my value throughput = 106.677'MB/sec zip pipeline throughput = 106.677'MB/sec entries = 7380822 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.48 avg-zval = 24.12 usrkeys = 7380822 avg-key = 20.00 avg-zkey = 1.91 avg-val = 117.48 avg-zval = 24.12 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 10 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0141 GB value = 0.1780 GB dict = 8.68 MB all = 0.2009 GB } UnZip/Zip{ index = 10.4523 value = 4.8702 dict = 1.00 all = 5.3452 } Zip/UnZip{ index = 0.0957 value = 0.2053 dict = 1.00 all = 0.1871 } ---------------------------- total value len = 28.617799 GB avg = 0.118 KB (by entry num) total key len = 6.815684 GB avg = 0.028 KB total ukey len = 4.868345 GB avg = 0.020 KB total ukey num = 0.243417273 Billion total entry num = 0.243417273 Billion write speed all = 11.083088655 MB/sec (with seq num) write speed all = 10.473989006 MB/sec (without seq num) 2024/07/08-17:44:08.417102 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634248: 7380822 keys, 200878867 bytes, temperature: kWarm 2024/07/08-17:44:08.417172 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431848417136, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634248, "file_size": 200878867, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 178036224, "index_size": 14122816, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206663016, "raw_average_key_size": 28, "raw_value_size": 867078892, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7380822, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431828, "oldest_key_time": 0, "file_creation_time": 1720431828, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634248, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:44:08.566409 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634249.sst) /dev/shm/ToplingTemp/Topling-17573-ejfQQZ 2024/07/08-17:44:13.922432 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634249.sst): first pass time = 5.36's, 200.476'MB/sec 2024/07/08-17:44:22.033148 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634249.sst): old prealloc_size = 1073741824, real_size = 187564416 2024/07/08-17:44:22.037329 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634249.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 7381030 prefix = 4 raw-key = 0.1181 GB zip-key = 0.0140 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.8671 GB zip-val = 0.1717 GB avg-val = 117.47 avg-zval = 23.26 2024/07/08-17:44:22.037385 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634249.sst: sumWaitingMem = 0.021 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0212 GB, waited 0.000 sec, Key+Value = 1.074 GB 2024/07/08-17:44:27.662259 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634249.sst): second pass time = 8.11's, 106.940'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 5.63's, 34.542'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3389.070'MB/sec (index lex order gen) rebuild zvType time = 4.62's, 0.400'MB/sec write SST data time = 1.01's, 192.561'MB/sec dict compress time = 0.00's, 8682336.000'MB/sec z-dict build time = 0.00's, sample length = 8.682'MB, throughput = 155980.382'MB/sec zip my value time = 8.11's, unzip length = 0.867'GB zip my value throughput = 106.940'MB/sec zip pipeline throughput = 106.940'MB/sec entries = 7381030 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.47 avg-zval = 23.26 usrkeys = 7381030 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.47 avg-zval = 23.26 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.1476 GB value = 0.8671 GB dict = 8.68 MB all = 1.0737 GB } __ZipSize{ index = 0.0140 GB value = 0.1717 GB dict = 8.68 MB all = 0.1944 GB } UnZip/Zip{ index = 10.5225 value = 5.0502 dict = 1.00 all = 5.5222 } Zip/UnZip{ index = 0.0950 value = 0.1980 dict = 1.00 all = 0.1811 } ---------------------------- total value len = 29.484872 GB avg = 0.118 KB (by entry num) total key len = 7.022352 GB avg = 0.028 KB total ukey len = 5.015966 GB avg = 0.020 KB total ukey num = 0.250798303 Billion total entry num = 0.250798303 Billion write speed all = 11.349808635 MB/sec (with seq num) write speed all = 10.726038892 MB/sec (without seq num) 2024/07/08-17:44:27.873167 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634249: 7381030 keys, 194439591 bytes, temperature: kWarm 2024/07/08-17:44:27.873236 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431867873202, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634249, "file_size": 194439591, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 171689968, "index_size": 14028992, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206668840, "raw_average_key_size": 28, "raw_value_size": 867073056, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7381030, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431848, "oldest_key_time": 0, "file_creation_time": 1720431848, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634249, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:44:28.017214 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634250.sst) /dev/shm/ToplingTemp/Topling-17573-k8R580 2024/07/08-17:44:33.837453 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634250.sst): first pass time = 5.82's, 184.486'MB/sec 2024/07/08-17:44:41.673809 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634250.sst): old prealloc_size = 1073741824, real_size = 226279584 2024/07/08-17:44:41.673884 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634250.sst): index type = FixedLenKeyIndex , store type = DictZipBlobStore usrkeys = 7357760 prefix = 4 raw-key = 0.1177 GB zip-key = 0.0633 GB avg-key = 16.00 avg-zkey = 8.61 raw-val = 0.8677 GB zip-val = 0.1611 GB avg-val = 117.93 avg-zval = 21.90 2024/07/08-17:44:42.548688 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634250.sst): second pass time = 7.83's, 110.769'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 266.497'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2415772.963'MB/sec (index lex order gen) rebuild zvType time = 0.15's, 12.225'MB/sec write SST data time = 0.72's, 321.860'MB/sec dict compress time = 0.00's, 8689862.000'MB/sec z-dict build time = 0.00's, sample length = 8.690'MB, throughput = 183877.399'MB/sec zip my value time = 7.83's, unzip length = 0.868'GB zip my value throughput = 110.769'MB/sec zip pipeline throughput = 110.769'MB/sec entries = 7357760 avg-key = 28.00 avg-zkey = 8.61 avg-val = 117.93 avg-zval = 21.90 usrkeys = 7357760 avg-key = 20.00 avg-zkey = 8.61 avg-val = 117.93 avg-zval = 21.90 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 5 UnZipSize{ index = 0.1472 GB value = 0.8677 GB dict = 8.69 MB all = 1.0737 GB } __ZipSize{ index = 0.0633 GB value = 0.1611 GB dict = 8.69 MB all = 0.2332 GB } UnZip/Zip{ index = 2.3236 value = 5.3860 dict = 1.00 all = 4.6053 } Zip/UnZip{ index = 0.4304 value = 0.1857 dict = 1.00 all = 0.2171 } ---------------------------- total value len = 30.352597 GB avg = 0.118 KB (by entry num) total key len = 7.228370 GB avg = 0.028 KB total ukey len = 5.163121 GB avg = 0.020 KB total ukey num = 0.258156063 Billion total entry num = 0.258156063 Billion write speed all = 11.629802988 MB/sec (with seq num) write speed all = 10.990691342 MB/sec (without seq num) 2024/07/08-17:44:42.804413 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634250: 7357760 keys, 233153273 bytes, temperature: kWarm 2024/07/08-17:44:42.804483 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431882804448, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634250, "file_size": 233153273, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 161108048, "index_size": 63331904, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 206017280, "raw_average_key_size": 28, "raw_value_size": 867724684, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 7357760, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-FixedLenKeyIndex-DictZipBlobStore", "creation_time": 1720431868, "oldest_key_time": 0, "file_creation_time": 1720431868, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634250, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:44:42.959656 140554168411712 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634251.sst) /dev/shm/ToplingTemp/Topling-17573-JipCpY 2024/07/08-17:44:45.571122 140554168411712 [top_zip_table_builder.cc:1344] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634251.sst): first pass time = 2.61's, 241.066'MB/sec 2024/07/08-17:44:52.353507 140554168411712 [top_zip_table_builder.cc:2764] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634251.sst): old prealloc_size = 1073741824, real_size = 148888112 2024/07/08-17:44:52.356044 140554168411712 [top_zip_table_builder.cc:1765] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634251.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 4347710 prefix = 4 raw-key = 0.0696 GB zip-key = 0.0083 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.5078 GB zip-val = 0.1395 GB avg-val = 116.79 avg-zval = 32.08 2024/07/08-17:44:52.356094 140554168411712 [top_zip_table_builder.cc:1295] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634251.sst: sumWaitingMem = 0.012 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0125 GB, waited 0.000 sec, Key+Value = 0.630 GB 2024/07/08-17:44:55.708667 140554168411712 [top_zip_table_builder.cc:3092] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00/12634251.sst): second pass time = 6.78's, 74.915'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 3.36's, 45.575'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3334.493'MB/sec (index lex order gen) rebuild zvType time = 2.73's, 0.397'MB/sec write SST data time = 0.62's, 247.416'MB/sec dict compress time = 0.00's, 5070032.000'MB/sec z-dict build time = 0.00's, sample length = 5.070'MB, throughput = 110438.965'MB/sec zip my value time = 6.78's, unzip length = 0.508'GB zip my value throughput = 74.915'MB/sec zip pipeline throughput = 74.915'MB/sec entries = 4347710 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.79 avg-zval = 32.08 usrkeys = 4347710 avg-key = 20.00 avg-zkey = 1.91 avg-val = 116.79 avg-zval = 32.08 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0870 GB value = 0.5078 GB dict = 5.07 MB all = 0.6295 GB } __ZipSize{ index = 0.0083 GB value = 0.1395 GB dict = 5.07 MB all = 0.1529 GB } UnZip/Zip{ index = 10.4519 value = 3.6405 dict = 1.00 all = 4.1169 } Zip/UnZip{ index = 0.0957 value = 0.2747 dict = 1.00 all = 0.2429 } ---------------------------- total value len = 30.860380 GB avg = 0.118 KB (by entry num) total key len = 7.350106 GB avg = 0.028 KB total ukey len = 5.250075 GB avg = 0.020 KB total ukey num = 0.262503773 Billion total entry num = 0.262503773 Billion write speed all = 11.776653706 MB/sec (with seq num) write speed all = 11.129414398 MB/sec (without seq num) 2024/07/08-17:44:55.884261 140554168411712 [compaction_job.cc:1944] [default] [JOB 8639] Generated table #12634251: 4347710 keys, 152911831 bytes, temperature: kWarm 2024/07/08-17:44:55.884332 140554168411712 EVENT_LOG_v1 {"time_micros": 1720431895884296, "cf_name": "default", "job": 8639, "event": "table_file_creation", "file_number": 12634251, "file_size": 152911831, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 139481568, "index_size": 8319424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 121735880, "raw_average_key_size": 28, "raw_value_size": 507783420, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 4347710, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1720431882, "oldest_key_time": 0, "file_creation_time": 1720431882, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a007a635-5c0a-4b4a-be96-b9c695cb016b", "db_session_id": "465QC0QPK7Q93426EREI", "orig_file_number": 12634251, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/07/08-17:44:55.974310 140554168411712 [compaction_job.cc:699] job-08639: subcompact[0], size: 7.215322 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08639/att-00 [12634216,12634217,12634218,12634219,12634220,12634221,12634222,12634223,12634224,12634225,12634226,12634227,12634228,12634229,12634230,12634231,12634232,12634233,12634234,12634235,12634236,12634237,12634238,12634239,12634240,12634241,12634242,12634243,12634244,12634245,12634246,12634247,12634248,12634249,12634250,12634251] 2024/07/08-17:45:12.202759 140554168411712 (Original Log Time 2024/07/08-17:32:53.763597) EVENT_LOG_v1 {"time_micros": 1720431173763543, "job": 8639, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L5": [12628932, 12628936, 12628937, 12628938, 12628939, 12628940, 12628941, 12628942, 12628943, 12628944, 12628945, 12628946, 12628947, 12628948, 12628949, 12628950, 12628951, 12628952, 12629150, 12629151], "files_L6": [12634212, 12634213, 12393294, 12393295, 12393296, 12393297, 12393298, 12393299, 12393300, 12393301, 12393302, 12393303, 12393304, 12393305, 12393306, 12393307, 12393308, 12393309, 12393310, 12393311, 12393312, 12393313, 12393314, 12393315, 12393316, 12393317, 12393318], "score": -1, "input_data_size": 7973076962, "oldest_snapshot_seqno": 147361293780} 2024/07/08-17:45:12.202767 140554168411712 (Original Log Time 2024/07/08-17:45:12.190008) [compaction_job.cc:2018] [default] [JOB 8639] Compacted 20@5 + 27@6 files to L6 => 7215321834 bytes 2024/07/08-17:45:12.202769 140554168411712 (Original Log Time 2024/07/08-17:45:12.202612) [compaction_job.cc:1220] [default] compacted to: files[0 0 0 0 0 0 36] max score 0.00, MB/sec: 11.0 rd, 10.0 wr, level 6, files in(20, 27) out(36 +0 blob) MB in(2354.4, 5249.4 +0.0 blob) out(6881.1 +0.0 blob), read-write-amplify(6.2) write-amplify(2.9) OK, records in: 262854241, records dropped: 350468 output_compression: Snappy 2024/07/08-17:45:12.202799 140554168411712 (Original Log Time 2024/07/08-17:45:12.202713) EVENT_LOG_v1 {"time_micros": 1720431912202653, "cf": "default", "job": 8639, "event": "compaction_finished", "compaction_time_micros": 722210718, "compaction_time_cpu_micros": 346584623, "output_level": 6, "num_output_files": 36, "total_output_size": 7215321834, "num_input_records": 262854241, "num_output_records": 262503773, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 4421139613, "file_range_sync_nanos": 0, "file_fsync_nanos": 28810, "file_prepare_write_nanos": 107712, "lsm_state": [0, 0, 0, 0, 0, 0, 36]} 2024/07/08-17:45:12.338002 140554168411712 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"a007a635-5c0a-4b4a-be96-b9c695cb016b","dbStarts":1720420896,"starts":1720431173,"executesMs":738817,"compactionJobId":8639,"attempt":0,"compactionInputRawBytes":38260252062,"compactionInputZipBytes":7973076962,"compactionOutputRawBytes":0,"compactionOutputZipBytes":7215321834,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-8cb3250d7fa4061690837ab3fd9dfd70-fb0e6dc5ed293055-00"}}, response = 2024/07/08-17:45:12.338116 140554168411712 INFO dcompact_worker.cpp:1456: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-07-08T14.41.36/.rocksdb/job-08639/att-00: olev 6, work 738.817 s, result 6.519 ms, install 12.842 ms, input{raw 35.633 GiB zip 7.426 GiB}