2024/11/23-02:56:47.249130 140024894502464 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00: fork to child time = 0.015487 sec 2024/11/23-02:56:47.250194 140024894502464 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00 2024/11/23-02:56:47.258685 140024894502464 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00 2024/11/23-02:56:47.260272 140024894502464 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-02:56:47.260603 140024894502464 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-02:56:47.309229 140024894502464 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-17846/att-00/MANIFEST-206347 2024/11/23-02:56:47.344770 140024894502464 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-17846/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 241685, last_sequence is 3746085730, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 241493 2024/11/23-02:56:47.344792 140024894502464 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-02:56:47.344928 140024894502464 [version_set.cc:5721] Creating manifest 241685 2024/11/23-02:56:47.377382 140024894502464 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-02:56:47.395730 140024894502464 [compaction_job.cc:2437] [default] [JOB 17846] Compacting 1@2 + 14@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-02:56:47.395746 140024894502464 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 13328 Base level 2, inputs: [241629(27MB)], [241650(35MB) 241651(1140KB) 241596(35MB) 241597(36MB) 241598(35MB) 241599(33MB) 241600(8730KB) 241577(35MB) 241578(35MB) 241579(35MB) 241580(33MB) 241581(36MB) 241582(37MB) 241676(35MB)] 2024/11/23-02:56:47.401499 140024894502464 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-02:56:47.401530 140024894502464 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-02:56:47.401550 140024894502464 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-02:56:47.401555 140024894502464 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-02:56:47.401566 140024894502464 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-02:56:47.983728 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241686.sst) /dev/shm/ToplingTemp/Topling-387648-HMqasj 2024/11/23-02:56:48.518054 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241686.sst): first pass time = 0.53's, 251.212'MB/sec 2024/11/23-02:56:49.881867 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241686.sst): old prealloc_size = 147639500, real_size = 33281744 2024/11/23-02:56:49.882430 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241686.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934192 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0313 GB avg-val = 115.67 avg-zval = 33.47 2024/11/23-02:56:49.883249 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241686.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:56:50.914024 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241686.sst): second pass time = 1.36's, 79.281'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.03's, 36.350'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3281.200'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.261'MB/sec write SST data time = 0.14's, 272.279'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 24003.918'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.281'MB/sec zip pipeline throughput = 79.281'MB/sec entries = 934192 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.67 avg-zval = 33.47 usrkeys = 934192 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.67 avg-zval = 33.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934192 bytes = 3386464 } seq expand size = 6539344 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5137 value = 3.4556 dict = 1.00 all = 3.5773 } Zip/UnZip{ index = 0.0951 value = 0.2894 dict = 1.00 all = 0.2795 } ---------------------------- total value len = 0.114600 GB avg = 0.123 KB (by entry num) total key len = 0.026157 GB avg = 0.028 KB total ukey len = 0.018684 GB avg = 0.020 KB total ukey num = 0.000934192 Billion total entry num = 0.000934192 Billion write speed all = 0.002163237 MB/sec (with seq num) write speed all = 0.002048379 MB/sec (without seq num) 2024/11/23-02:56:50.947274 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241686: 934192 keys, 37518808 bytes, temperature: kWarm 2024/11/23-02:56:50.947377 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301810947313, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241686, "file_size": 37518808, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3745834686, "table_properties": {"data_size": 31270912, "index_size": 1777088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26157376, "raw_average_key_size": 28, "raw_value_size": 108060376, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934192, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301807, "oldest_key_time": 0, "file_creation_time": 1732301807, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241686, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:56:50.996607 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241687.sst) /dev/shm/ToplingTemp/Topling-387648-hMcmij 2024/11/23-02:56:51.912589 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241687.sst): first pass time = 0.92's, 146.534'MB/sec 2024/11/23-02:56:53.291349 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241687.sst): old prealloc_size = 147639500, real_size = 32982768 2024/11/23-02:56:53.291896 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241687.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926029 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0310 GB avg-val = 116.94 avg-zval = 33.47 2024/11/23-02:56:53.292650 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241687.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:56:54.348809 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241687.sst): second pass time = 1.38's, 78.578'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 35.065'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3359.021'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.256'MB/sec write SST data time = 0.15's, 244.942'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34458.614'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.578'MB/sec zip pipeline throughput = 78.578'MB/sec entries = 926029 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.94 avg-zval = 33.47 usrkeys = 926029 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.94 avg-zval = 33.47 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926029 bytes = 3241120 } seq expand size = 6482203 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0371 GB } UnZip/Zip{ index = 10.5169 value = 3.4943 dict = 1.00 all = 3.6198 } Zip/UnZip{ index = 0.0951 value = 0.2862 dict = 1.00 all = 0.2763 } ---------------------------- total value len = 0.229371 GB avg = 0.123 KB (by entry num) total key len = 0.052086 GB avg = 0.028 KB total ukey len = 0.037204 GB avg = 0.020 KB total ukey num = 0.001860221 Billion total entry num = 0.001860221 Billion write speed all = 0.004325368 MB/sec (with seq num) write speed all = 0.004096668 MB/sec (without seq num) 2024/11/23-02:56:54.381033 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241687: 926029 keys, 37078536 bytes, temperature: kWarm 2024/11/23-02:56:54.381101 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301814381071, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241687, "file_size": 37078536, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3745835374, "table_properties": {"data_size": 30990048, "index_size": 1761024, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928812, "raw_average_key_size": 28, "raw_value_size": 108288990, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926029, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301810, "oldest_key_time": 0, "file_creation_time": 1732301810, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241687, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:56:54.416890 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241688.sst) /dev/shm/ToplingTemp/Topling-387648-vudGej 2024/11/23-02:56:55.288660 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241688.sst): first pass time = 0.87's, 153.966'MB/sec 2024/11/23-02:56:56.647907 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241688.sst): old prealloc_size = 147639500, real_size = 35077888 2024/11/23-02:56:56.648492 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241688.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 930741 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0331 GB avg-val = 116.21 avg-zval = 35.52 2024/11/23-02:56:56.649075 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241688.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:56:57.576025 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241688.sst): second pass time = 1.36's, 79.612'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 41.142'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3175.497'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.291'MB/sec write SST data time = 0.13's, 297.057'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35538.892'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.612'MB/sec zip pipeline throughput = 79.612'MB/sec entries = 930741 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.21 avg-zval = 35.52 usrkeys = 930741 avg-key = 20.00 avg-zkey = 1.91 avg-val = 120.73 avg-zval = 35.52 TagRS{ kinds = RS_Key0_TagN bytes = 145520 } TagArray{ size = 601249 bytes = 2104384 } seq expand size = 4208743 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0331 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4459 value = 3.2712 dict = 1.00 all = 3.5151 } Zip/UnZip{ index = 0.0957 value = 0.3057 dict = 1.00 all = 0.2845 } ---------------------------- total value len = 0.341737 GB avg = 0.122 KB (by entry num) total key len = 0.078147 GB avg = 0.028 KB total ukey len = 0.055819 GB avg = 0.020 KB total ukey num = 0.002790962 Billion total entry num = 0.002790962 Billion write speed all = 0.006452355 MB/sec (with seq num) write speed all = 0.006109245 MB/sec (without seq num) 2024/11/23-02:56:57.608923 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241688: 930741 keys, 38183432 bytes, temperature: kWarm 2024/11/23-02:56:57.608984 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301817608953, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241688, "file_size": 38183432, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3745835428, "table_properties": {"data_size": 33063024, "index_size": 1782016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26060748, "raw_average_key_size": 28, "raw_value_size": 108157062, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 930741, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301814, "oldest_key_time": 0, "file_creation_time": 1732301814, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241688, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:56:57.645750 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241689.sst) /dev/shm/ToplingTemp/Topling-387648-yrxthj 2024/11/23-02:56:58.462741 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241689.sst): first pass time = 0.82's, 164.302'MB/sec 2024/11/23-02:56:59.892265 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241689.sst): old prealloc_size = 147639500, real_size = 35182448 2024/11/23-02:56:59.892932 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241689.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081409 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0328 GB avg-val = 96.11 avg-zval = 30.37 2024/11/23-02:56:59.893105 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241689.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:57:00.637077 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241689.sst): second pass time = 1.43's, 72.749'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.74's, 48.591'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3213.481'MB/sec (index lex order gen) rebuild zvType time = 0.61's, 0.445'MB/sec write SST data time = 0.14's, 264.810'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.034'MB, throughput = 31451.325'MB/sec zip my value time = 1.43's, unzip length = 0.104'GB zip my value throughput = 72.749'MB/sec zip pipeline throughput = 72.749'MB/sec entries = 1081409 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.11 avg-zval = 30.37 usrkeys = 1081409 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.26 avg-zval = 30.37 TagRS{ kinds = RS_Key0_TagN bytes = 169080 } TagArray{ size = 22498 bytes = 64704 } seq expand size = 157486 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0328 GB dict = 1.03 MB all = 0.0362 GB } UnZip/Zip{ index = 10.4473 value = 3.1648 dict = 1.00 all = 3.7087 } Zip/UnZip{ index = 0.0957 value = 0.3160 dict = 1.00 all = 0.2696 } ---------------------------- total value len = 0.445832 GB avg = 0.115 KB (by entry num) total key len = 0.108426 GB avg = 0.028 KB total ukey len = 0.077447 GB avg = 0.020 KB total ukey num = 0.003872371 Billion total entry num = 0.003872371 Billion write speed all = 0.008516899 MB/sec (with seq num) write speed all = 0.008040868 MB/sec (without seq num) 2024/11/23-02:57:00.669733 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241689: 1081409 keys, 36190337 bytes, temperature: kWarm 2024/11/23-02:57:00.669804 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301820669764, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241689, "file_size": 36190337, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3277976803, "table_properties": {"data_size": 32841696, "index_size": 2070208, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30279452, "raw_average_key_size": 28, "raw_value_size": 103938279, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081409, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301817, "oldest_key_time": 0, "file_creation_time": 1732301817, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241689, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:57:00.706125 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241690.sst) /dev/shm/ToplingTemp/Topling-387648-0RWJpk 2024/11/23-02:57:01.593839 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241690.sst): first pass time = 0.89's, 151.201'MB/sec 2024/11/23-02:57:02.926173 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241690.sst): old prealloc_size = 147639500, real_size = 30897584 2024/11/23-02:57:02.926741 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241690.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926027 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0289 GB avg-val = 116.94 avg-zval = 31.20 2024/11/23-02:57:02.927643 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241690.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:57:03.909717 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241690.sst): second pass time = 1.33's, 81.323'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.98's, 35.696'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3237.827'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.291'MB/sec write SST data time = 0.19's, 186.711'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32700.051'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.323'MB/sec zip pipeline throughput = 81.323'MB/sec entries = 926027 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.94 avg-zval = 31.20 usrkeys = 926027 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.94 avg-zval = 31.20 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926027 bytes = 3356864 } seq expand size = 6482189 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0289 GB dict = 1.08 MB all = 0.0351 GB } UnZip/Zip{ index = 10.4444 value = 3.7480 dict = 1.00 all = 3.8230 } Zip/UnZip{ index = 0.0957 value = 0.2668 dict = 1.00 all = 0.2616 } ---------------------------- total value len = 0.560604 GB avg = 0.117 KB (by entry num) total key len = 0.134355 GB avg = 0.028 KB total ukey len = 0.095968 GB avg = 0.020 KB total ukey num = 0.004798398 Billion total entry num = 0.004798398 Billion write speed all = 0.010678398 MB/sec (with seq num) write speed all = 0.010088559 MB/sec (without seq num) 2024/11/23-02:57:03.944141 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241690: 926027 keys, 35107664 bytes, temperature: kWarm 2024/11/23-02:57:03.944201 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301823944170, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241690, "file_size": 35107664, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3277976804, "largest_seqno": 3745834298, "table_properties": {"data_size": 28892640, "index_size": 1773248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25928756, "raw_average_key_size": 28, "raw_value_size": 108288982, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926027, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301820, "oldest_key_time": 0, "file_creation_time": 1732301820, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241690, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:57:03.979566 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241691.sst) /dev/shm/ToplingTemp/Topling-387648-OnI3Gk 2024/11/23-02:57:04.950153 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241691.sst): first pass time = 0.97's, 138.290'MB/sec 2024/11/23-02:57:06.267714 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241691.sst): old prealloc_size = 147639500, real_size = 31357424 2024/11/23-02:57:06.268275 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241691.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925935 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0294 GB avg-val = 116.95 avg-zval = 31.70 2024/11/23-02:57:06.269043 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241691.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:57:07.260523 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241691.sst): second pass time = 1.32's, 82.230'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.99's, 35.711'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3304.327'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.20's, 181.645'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35090.626'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.230'MB/sec zip pipeline throughput = 82.230'MB/sec entries = 925935 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.95 avg-zval = 31.70 usrkeys = 925935 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.95 avg-zval = 31.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925935 bytes = 3240784 } seq expand size = 6481545 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0294 GB dict = 1.08 MB all = 0.0355 GB } UnZip/Zip{ index = 10.4426 value = 3.6894 dict = 1.00 all = 3.7858 } Zip/UnZip{ index = 0.0958 value = 0.2710 dict = 1.00 all = 0.2641 } ---------------------------- total value len = 0.675377 GB avg = 0.118 KB (by entry num) total key len = 0.160281 GB avg = 0.028 KB total ukey len = 0.114487 GB avg = 0.020 KB total ukey num = 0.005724333 Billion total entry num = 0.005724333 Billion write speed all = 0.012839654 MB/sec (with seq num) write speed all = 0.012136032 MB/sec (without seq num) 2024/11/23-02:57:07.292928 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241691: 925935 keys, 35453246 bytes, temperature: kWarm 2024/11/23-02:57:07.292985 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301827292956, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241691, "file_size": 35453246, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3745836263, "table_properties": {"data_size": 29352352, "index_size": 1773376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25926180, "raw_average_key_size": 28, "raw_value_size": 108291654, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925935, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301823, "oldest_key_time": 0, "file_creation_time": 1732301823, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241691, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:57:07.328877 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241692.sst) /dev/shm/ToplingTemp/Topling-387648-QXIn6i 2024/11/23-02:57:08.112294 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241692.sst): first pass time = 0.78's, 171.333'MB/sec 2024/11/23-02:57:09.486145 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241692.sst): old prealloc_size = 147639500, real_size = 33900096 2024/11/23-02:57:09.486729 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241692.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929288 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1082 GB zip-val = 0.0319 GB avg-val = 116.43 avg-zval = 34.31 2024/11/23-02:57:09.487550 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241692.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:57:10.531921 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241692.sst): second pass time = 1.37's, 78.794'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.05's, 36.454'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3154.918'MB/sec (index lex order gen) rebuild zvType time = 0.89's, 0.262'MB/sec write SST data time = 0.16's, 238.399'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31552.736'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 78.794'MB/sec zip pipeline throughput = 78.794'MB/sec entries = 929288 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.43 avg-zval = 34.31 usrkeys = 929288 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.43 avg-zval = 34.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929288 bytes = 3368688 } seq expand size = 6505016 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0381 GB } UnZip/Zip{ index = 10.4435 value = 3.3931 dict = 1.00 all = 3.5208 } Zip/UnZip{ index = 0.0958 value = 0.2947 dict = 1.00 all = 0.2840 } ---------------------------- total value len = 0.790080 GB avg = 0.119 KB (by entry num) total key len = 0.186301 GB avg = 0.028 KB total ukey len = 0.133072 GB avg = 0.020 KB total ukey num = 0.006653621 Billion total entry num = 0.006653621 Billion write speed all = 0.015001066 MB/sec (with seq num) write speed all = 0.014183259 MB/sec (without seq num) 2024/11/23-02:57:10.565512 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241692: 929288 keys, 38121612 bytes, temperature: kWarm 2024/11/23-02:57:10.565574 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301830565542, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241692, "file_size": 38121612, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3745836201, "table_properties": {"data_size": 31887920, "index_size": 1779648, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26020064, "raw_average_key_size": 28, "raw_value_size": 108197676, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929288, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301827, "oldest_key_time": 0, "file_creation_time": 1732301827, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241692, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:57:10.600876 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241693.sst) /dev/shm/ToplingTemp/Topling-387648-79dhgk 2024/11/23-02:57:11.504963 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241693.sst): first pass time = 0.90's, 148.462'MB/sec 2024/11/23-02:57:12.864557 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241693.sst): old prealloc_size = 147639500, real_size = 33048832 2024/11/23-02:57:12.865116 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241693.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927916 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0311 GB avg-val = 116.64 avg-zval = 33.46 2024/11/23-02:57:12.865845 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241693.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:57:13.902883 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241693.sst): second pass time = 1.36's, 79.656'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 35.556'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3283.669'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.258'MB/sec write SST data time = 0.14's, 264.165'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34393.326'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.656'MB/sec zip pipeline throughput = 79.656'MB/sec entries = 927916 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.64 avg-zval = 33.46 usrkeys = 927916 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.64 avg-zval = 33.46 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927916 bytes = 3015744 } seq expand size = 6495412 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5124 value = 3.4857 dict = 1.00 all = 3.6355 } Zip/UnZip{ index = 0.0951 value = 0.2869 dict = 1.00 all = 0.2751 } ---------------------------- total value len = 0.904811 GB avg = 0.119 KB (by entry num) total key len = 0.212283 GB avg = 0.028 KB total ukey len = 0.151631 GB avg = 0.020 KB total ukey num = 0.007581537 Billion total entry num = 0.007581537 Billion write speed all = 0.017162089 MB/sec (with seq num) write speed all = 0.016230278 MB/sec (without seq num) 2024/11/23-02:57:13.935810 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241693: 927916 keys, 36918568 bytes, temperature: kWarm 2024/11/23-02:57:13.935868 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301833935839, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241693, "file_size": 36918568, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3680980241, "largest_seqno": 3745835485, "table_properties": {"data_size": 31051312, "index_size": 1765376, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25981648, "raw_average_key_size": 28, "raw_value_size": 108236204, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927916, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301830, "oldest_key_time": 0, "file_creation_time": 1732301830, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241693, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:57:13.983489 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241694.sst) /dev/shm/ToplingTemp/Topling-387648-HmywPk 2024/11/23-02:57:14.800872 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241694.sst): first pass time = 0.82's, 164.211'MB/sec 2024/11/23-02:57:16.153514 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241694.sst): old prealloc_size = 147639500, real_size = 32948368 2024/11/23-02:57:16.154068 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241694.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927471 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0310 GB avg-val = 116.71 avg-zval = 33.37 2024/11/23-02:57:16.155596 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241694.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:57:17.107644 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241694.sst): second pass time = 1.35's, 80.068'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.952'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3313.683'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.281'MB/sec write SST data time = 0.13's, 288.155'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 38384.522'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.068'MB/sec zip pipeline throughput = 80.068'MB/sec entries = 927471 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.71 avg-zval = 33.37 usrkeys = 927471 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.71 avg-zval = 33.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927471 bytes = 3362096 } seq expand size = 6492297 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0310 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5127 value = 3.4973 dict = 1.00 all = 3.6114 } Zip/UnZip{ index = 0.0951 value = 0.2859 dict = 1.00 all = 0.2769 } ---------------------------- total value len = 1.019552 GB avg = 0.120 KB (by entry num) total key len = 0.238252 GB avg = 0.028 KB total ukey len = 0.170180 GB avg = 0.020 KB total ukey num = 0.008509008 Billion total entry num = 0.008509008 Billion write speed all = 0.019322887 MB/sec (with seq num) write speed all = 0.018277137 MB/sec (without seq num) 2024/11/23-02:57:17.143310 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241694: 927471 keys, 37164662 bytes, temperature: kWarm 2024/11/23-02:57:17.143372 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301837143339, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241694, "file_size": 37164662, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369194706, "largest_seqno": 3745835570, "table_properties": {"data_size": 30951808, "index_size": 1764480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25969188, "raw_average_key_size": 28, "raw_value_size": 108248554, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927471, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301833, "oldest_key_time": 0, "file_creation_time": 1732301833, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241694, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:57:17.179500 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241695.sst) /dev/shm/ToplingTemp/Topling-387648-3HcUci 2024/11/23-02:57:18.089714 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241695.sst): first pass time = 0.91's, 147.464'MB/sec 2024/11/23-02:57:19.421622 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241695.sst): old prealloc_size = 147639500, real_size = 33494224 2024/11/23-02:57:19.422194 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241695.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934654 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1080 GB zip-val = 0.0315 GB avg-val = 115.60 avg-zval = 33.66 2024/11/23-02:57:19.423017 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241695.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:57:20.341311 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241695.sst): second pass time = 1.33's, 81.167'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.028'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3274.547'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.296'MB/sec write SST data time = 0.13's, 290.864'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 31438.371'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.167'MB/sec zip pipeline throughput = 81.167'MB/sec entries = 934654 avg-key = 28.00 avg-zkey = 1.93 avg-val = 115.60 avg-zval = 33.66 usrkeys = 934654 avg-key = 20.00 avg-zkey = 1.93 avg-val = 122.60 avg-zval = 33.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934654 bytes = 3388128 } seq expand size = 6542578 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.3758 value = 3.4346 dict = 1.00 all = 3.5571 } Zip/UnZip{ index = 0.0964 value = 0.2912 dict = 1.00 all = 0.2811 } ---------------------------- total value len = 1.134142 GB avg = 0.120 KB (by entry num) total key len = 0.264423 GB avg = 0.028 KB total ukey len = 0.188873 GB avg = 0.020 KB total ukey num = 0.009443662 Billion total entry num = 0.009443662 Billion write speed all = 0.021484237 MB/sec (with seq num) write speed all = 0.020323676 MB/sec (without seq num) 2024/11/23-02:57:20.374406 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241695: 934654 keys, 37732504 bytes, temperature: kWarm 2024/11/23-02:57:20.374467 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301840374434, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241695, "file_size": 37732504, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3339428666, "largest_seqno": 3745835935, "table_properties": {"data_size": 31458752, "index_size": 1801600, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26170312, "raw_average_key_size": 28, "raw_value_size": 108047492, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934654, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301837, "oldest_key_time": 0, "file_creation_time": 1732301837, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241695, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:57:20.415193 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241696.sst) /dev/shm/ToplingTemp/Topling-387648-vstfri 2024/11/23-02:57:21.245131 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241696.sst): first pass time = 0.83's, 161.726'MB/sec 2024/11/23-02:57:22.589526 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241696.sst): old prealloc_size = 147639500, real_size = 34364208 2024/11/23-02:57:22.590089 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241696.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929411 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0324 GB avg-val = 116.41 avg-zval = 34.82 2024/11/23-02:57:22.590914 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241696.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:57:23.504868 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241696.sst): second pass time = 1.34's, 80.522'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 42.155'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3252.328'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.296'MB/sec write SST data time = 0.13's, 295.255'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33902.278'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.522'MB/sec zip pipeline throughput = 80.522'MB/sec entries = 929411 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.41 avg-zval = 34.82 usrkeys = 929411 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.41 avg-zval = 34.82 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929411 bytes = 3369136 } seq expand size = 6505877 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0386 GB } UnZip/Zip{ index = 10.5107 value = 3.3431 dict = 1.00 all = 3.4784 } Zip/UnZip{ index = 0.0951 value = 0.2991 dict = 1.00 all = 0.2875 } ---------------------------- total value len = 1.248842 GB avg = 0.120 KB (by entry num) total key len = 0.290446 GB avg = 0.028 KB total ukey len = 0.207461 GB avg = 0.020 KB total ukey num = 0.010373073 Billion total entry num = 0.010373073 Billion write speed all = 0.023644832 MB/sec (with seq num) write speed all = 0.022370115 MB/sec (without seq num) 2024/11/23-02:57:23.538936 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241696: 929411 keys, 38585772 bytes, temperature: kWarm 2024/11/23-02:57:23.539003 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301843538968, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241696, "file_size": 38585772, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3745835172, "table_properties": {"data_size": 32363168, "index_size": 1768512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26023508, "raw_average_key_size": 28, "raw_value_size": 108194226, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929411, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301840, "oldest_key_time": 0, "file_creation_time": 1732301840, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241696, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:57:23.584475 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241697.sst) /dev/shm/ToplingTemp/Topling-387648-6yUopk 2024/11/23-02:57:24.466553 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241697.sst): first pass time = 0.88's, 152.167'MB/sec 2024/11/23-02:57:25.802549 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241697.sst): old prealloc_size = 147639500, real_size = 35528512 2024/11/23-02:57:25.803125 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241697.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928805 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.51 avg-zval = 36.09 2024/11/23-02:57:25.803836 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241697.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-02:57:26.660402 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241697.sst): second pass time = 1.33's, 81.075'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 45.736'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3228.594'MB/sec (index lex order gen) rebuild zvType time = 0.72's, 0.321'MB/sec write SST data time = 0.13's, 294.796'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31247.271'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.075'MB/sec zip pipeline throughput = 81.075'MB/sec entries = 928805 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.51 avg-zval = 36.09 usrkeys = 928805 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.13 avg-zval = 36.09 TagRS{ kinds = RS_Key0_TagN bytes = 145240 } TagArray{ size = 745808 bytes = 2703568 } seq expand size = 5220656 multi value expand size = 0 cnt WriteAppend = 17 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0392 GB } UnZip/Zip{ index = 10.4452 value = 3.2285 dict = 1.00 all = 3.4210 } Zip/UnZip{ index = 0.0957 value = 0.3097 dict = 1.00 all = 0.2923 } ---------------------------- total value len = 1.362274 GB avg = 0.121 KB (by entry num) total key len = 0.316453 GB avg = 0.028 KB total ukey len = 0.226038 GB avg = 0.020 KB total ukey num = 0.011301878 Billion total entry num = 0.011301878 Billion write speed all = 0.025785480 MB/sec (with seq num) write speed all = 0.024396692 MB/sec (without seq num) 2024/11/23-02:57:26.695290 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241697: 928805 keys, 39234016 bytes, temperature: kWarm 2024/11/23-02:57:26.695352 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301846695320, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241697, "file_size": 39234016, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3745836309, "table_properties": {"data_size": 33517680, "index_size": 1778432, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26006540, "raw_average_key_size": 28, "raw_value_size": 108211218, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928805, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301843, "oldest_key_time": 0, "file_creation_time": 1732301843, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241697, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:57:26.732381 140024894502464 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241698.sst) /dev/shm/ToplingTemp/Topling-387648-W59lSk 2024/11/23-02:57:26.959143 140024894502464 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241698.sst): first pass time = 0.23's, 251.962'MB/sec 2024/11/23-02:57:27.515273 140024894502464 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241698.sst): old prealloc_size = 147639500, real_size = 14589920 2024/11/23-02:57:27.515548 140024894502464 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241698.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 396484 prefix = 4 raw-key = 0.0063 GB zip-key = 0.0008 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.0460 GB zip-val = 0.0137 GB avg-val = 116.08 avg-zval = 34.63 2024/11/23-02:57:27.515915 140024894502464 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241698.sst: sumWaitingMem = 0.001 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0009 GB, waited 0.000 sec, Key+Value = 0.057 GB 2024/11/23-02:57:27.923077 140024894502464 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00/241698.sst): second pass time = 0.56's, 82.864'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.41's, 39.850'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3036.649'MB/sec (index lex order gen) rebuild zvType time = 0.35's, 0.281'MB/sec write SST data time = 0.05's, 296.083'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.469'MB, throughput = 14956.908'MB/sec zip my value time = 0.56's, unzip length = 0.046'GB zip my value throughput = 82.864'MB/sec zip pipeline throughput = 82.864'MB/sec entries = 396484 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.08 avg-zval = 34.63 usrkeys = 396484 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.08 avg-zval = 34.63 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 396484 bytes = 1288592 } seq expand size = 2775388 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0079 GB value = 0.0460 GB dict = 0.47 MB all = 0.0571 GB } __ZipSize{ index = 0.0008 GB value = 0.0137 GB dict = 0.47 MB all = 0.0163 GB } UnZip/Zip{ index = 10.4320 value = 3.3520 dict = 1.00 all = 3.5154 } Zip/UnZip{ index = 0.0959 value = 0.2983 dict = 1.00 all = 0.2845 } ---------------------------- total value len = 1.411075 GB avg = 0.121 KB (by entry num) total key len = 0.327554 GB avg = 0.028 KB total ukey len = 0.233967 GB avg = 0.020 KB total ukey num = 0.011698362 Billion total entry num = 0.011698362 Billion write speed all = 0.026705067 MB/sec (with seq num) write speed all = 0.025267587 MB/sec (without seq num) 2024/11/23-02:57:27.942964 140024894502464 [compaction_job.cc:1948] [default] [JOB 17846] Generated table #241698: 396484 keys, 16250345 bytes, temperature: kWarm 2024/11/23-02:57:27.943023 140024894502464 EVENT_LOG_v1 {"time_micros": 1732301847942991, "cf_name": "default", "job": 17846, "event": "table_file_creation", "file_number": 241698, "file_size": 16250345, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3696165216, "largest_seqno": 3745832642, "table_properties": {"data_size": 13730512, "index_size": 760128, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 11101552, "raw_average_key_size": 28, "raw_value_size": 46025200, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 396484, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732301846, "oldest_key_time": 0, "file_creation_time": 1732301846, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 241698, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-02:57:27.960257 140024894502464 [compaction_job.cc:703] job-17846: subcompact[0], size: 0.463540 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-17846/att-00 [241686,241687,241688,241689,241690,241691,241692,241693,241694,241695,241696,241697,241698] 2024/11/23-02:57:28.053451 140024894502464 (Original Log Time 2024/11/23-02:56:47.395806) EVENT_LOG_v1 {"time_micros": 1732301807395760, "job": 17846, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [241629], "files_L3": [241650, 241651, 241596, 241597, 241598, 241599, 241600, 241577, 241578, 241579, 241580, 241581, 241582, 241676], "score": 0.99488, "input_data_size": 484129429, "oldest_snapshot_seqno": 3746085698} 2024/11/23-02:57:28.053454 140024894502464 (Original Log Time 2024/11/23-02:57:28.052754) [compaction_job.cc:2022] [default] [JOB 17846] Compacted 1@2 + 14@3 files to L3 => 463539502 bytes 2024/11/23-02:57:28.053456 140024894502464 (Original Log Time 2024/11/23-02:57:28.053378) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 13 0 0 0] max score 0.02, MB/sec: 11.9 rd, 11.4 wr, level 3, files in(1, 14) out(13 +0 blob) MB in(27.1, 434.6 +0.0 blob) out(442.1 +0.0 blob), read-write-amplify(33.3) write-amplify(16.3) OK, records in: 11754606, records dropped: 56244 output_compression: Snappy 2024/11/23-02:57:28.053462 140024894502464 (Original Log Time 2024/11/23-02:57:28.053430) EVENT_LOG_v1 {"time_micros": 1732301848053409, "cf": "default", "job": 17846, "event": "compaction_finished", "compaction_time_micros": 40564458, "compaction_time_cpu_micros": 18542635, "output_level": 3, "num_output_files": 13, "total_output_size": 463539502, "num_input_records": 11754606, "num_output_records": 11698362, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 327086244, "file_range_sync_nanos": 0, "file_fsync_nanos": 11085, "file_prepare_write_nanos": 31755, "lsm_state": [0, 0, 0, 13, 0, 0, 0]} 2024/11/23-02:57:28.180137 140024894502464 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732301807,"executesMs":40797,"compactionJobId":17846,"attempt":0,"compactionInputRawBytes":1675726697,"compactionInputZipBytes":484129429,"compactionOutputRawBytes":0,"compactionOutputZipBytes":463539502,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-a40d1cb3f7e2d54b3d9130cca658d623-3084d353463e28d2-00"}}, response = 2024/11/23-02:57:28.180208 140024894502464 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-17846/att-00: olev 3, work 40.797 s, result 6.315 ms, install 0.726 ms, input{raw 1.561 GiB zip 461.702 MiB}