2024/11/22-22:48:41.186560 140024928073280 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00: fork to child time = 0.023294 sec 2024/11/22-22:48:41.187800 140024928073280 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00 2024/11/22-22:48:41.195974 140024928073280 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00 2024/11/22-22:48:41.197114 140024928073280 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-22:48:41.197533 140024928073280 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-22:48:41.252006 140024928073280 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13174/att-00/MANIFEST-206347 2024/11/22-22:48:41.265863 140024928073280 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13174/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 232187, last_sequence is 3621161115, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 232024 2024/11/22-22:48:41.265873 140024928073280 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-22:48:41.266005 140024928073280 [version_set.cc:5721] Creating manifest 232187 2024/11/22-22:48:41.299018 140024928073280 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-22:48:41.319305 140024928073280 [compaction_job.cc:2437] [default] [JOB 13174] Compacting 1@2 + 15@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/22-22:48:41.319330 140024928073280 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 9913 Base level 2, inputs: [232182(27MB)], [232109(35MB) 232110(35MB) 232111(35MB) 232112(34MB) 232113(35MB) 232114(35MB) 232115(35MB) 232116(35MB) 232117(38MB) 232118(35MB) 232119(35MB) 232120(33MB) 232121(35MB) 232122(36MB) 232123(33MB)] 2024/11/22-22:48:41.327084 140024928073280 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:48:41.327124 140024928073280 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:48:41.327152 140024928073280 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-22:48:41.327160 140024928073280 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:48:41.327174 140024928073280 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-22:48:41.932631 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232188.sst) /dev/shm/ToplingTemp/Topling-321443-6DbMNd 2024/11/22-22:48:42.817979 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232188.sst): first pass time = 0.89's, 151.608'MB/sec 2024/11/22-22:48:46.684811 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232188.sst): old prealloc_size = 147639500, real_size = 32887488 2024/11/22-22:48:46.685396 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232188.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924912 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0309 GB avg-val = 117.11 avg-zval = 33.41 2024/11/22-22:48:46.686161 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232188.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:48:47.917113 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232188.sst): second pass time = 3.86's, 28.048'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.23's, 29.915'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3155.212'MB/sec (index lex order gen) rebuild zvType time = 1.06's, 0.218'MB/sec write SST data time = 0.17's, 215.130'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 26612.097'MB/sec zip my value time = 3.86's, unzip length = 0.108'GB zip my value throughput = 28.048'MB/sec zip pipeline throughput = 125.436'MB/sec entries = 924912 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.11 avg-zval = 33.41 usrkeys = 924912 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.11 avg-zval = 33.41 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924912 bytes = 3121600 } seq expand size = 6474384 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0309 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.5146 value = 3.5059 dict = 1.00 all = 3.6408 } Zip/UnZip{ index = 0.0951 value = 0.2852 dict = 1.00 all = 0.2747 } ---------------------------- total value len = 0.114795 GB avg = 0.124 KB (by entry num) total key len = 0.025898 GB avg = 0.028 KB total ukey len = 0.018498 GB avg = 0.020 KB total ukey num = 0.000924912 Billion total entry num = 0.000924912 Billion write speed all = 0.002803482 MB/sec (with seq num) write speed all = 0.002656041 MB/sec (without seq num) 2024/11/22-22:48:47.956958 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232188: 924912 keys, 36864706 bytes, temperature: kWarm 2024/11/22-22:48:47.957047 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286927956994, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232188, "file_size": 36864706, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3513012344, "largest_seqno": 3621074125, "table_properties": {"data_size": 30896752, "index_size": 1759296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25897536, "raw_average_key_size": 28, "raw_value_size": 108320312, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924912, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286921, "oldest_key_time": 0, "file_creation_time": 1732286921, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232188, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:48:47.995497 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232189.sst) /dev/shm/ToplingTemp/Topling-321443-1AC5ld 2024/11/22-22:48:49.366325 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232189.sst): first pass time = 1.37's, 97.912'MB/sec 2024/11/22-22:48:51.137109 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232189.sst): old prealloc_size = 147639500, real_size = 33588560 2024/11/22-22:48:51.137676 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232189.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924664 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 117.15 avg-zval = 34.17 2024/11/22-22:48:51.138462 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232189.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:48:52.450159 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232189.sst): second pass time = 1.77's, 61.216'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.31's, 28.698'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3277.590'MB/sec (index lex order gen) rebuild zvType time = 1.09's, 0.213'MB/sec write SST data time = 0.23's, 166.114'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 37907.405'MB/sec zip my value time = 1.77's, unzip length = 0.108'GB zip my value throughput = 61.216'MB/sec zip pipeline throughput = 61.216'MB/sec entries = 924664 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 34.17 usrkeys = 924664 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 34.17 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924664 bytes = 3236336 } seq expand size = 6472648 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5125 value = 3.4283 dict = 1.00 all = 3.5619 } Zip/UnZip{ index = 0.0951 value = 0.2917 dict = 1.00 all = 0.2807 } ---------------------------- total value len = 0.229595 GB avg = 0.124 KB (by entry num) total key len = 0.051788 GB avg = 0.028 KB total ukey len = 0.036992 GB avg = 0.020 KB total ukey num = 0.001849576 Billion total entry num = 0.001849576 Billion write speed all = 0.005606422 MB/sec (with seq num) write speed all = 0.005311606 MB/sec (without seq num) 2024/11/22-22:48:52.490892 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232189: 924664 keys, 37681116 bytes, temperature: kWarm 2024/11/22-22:48:52.490969 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286932490927, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232189, "file_size": 37681116, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038037, "largest_seqno": 3621075165, "table_properties": {"data_size": 31598016, "index_size": 1759168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25890592, "raw_average_key_size": 28, "raw_value_size": 108327216, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924664, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286927, "oldest_key_time": 0, "file_creation_time": 1732286927, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232189, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:48:52.556700 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232190.sst) /dev/shm/ToplingTemp/Topling-321443-INunFb 2024/11/22-22:48:54.046195 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232190.sst): first pass time = 1.49's, 90.113'MB/sec 2024/11/22-22:48:55.716179 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232190.sst): old prealloc_size = 147639500, real_size = 33405264 2024/11/22-22:48:55.716734 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232190.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926123 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.92 avg-zval = 33.92 2024/11/22-22:48:55.717508 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232190.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:48:56.843742 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232190.sst): second pass time = 1.67's, 64.900'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.13's, 33.157'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3294.843'MB/sec (index lex order gen) rebuild zvType time = 1.00's, 0.232'MB/sec write SST data time = 0.13's, 294.410'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34735.167'MB/sec zip my value time = 1.67's, unzip length = 0.108'GB zip my value throughput = 64.900'MB/sec zip pipeline throughput = 64.900'MB/sec entries = 926123 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.92 avg-zval = 33.92 usrkeys = 926123 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.92 avg-zval = 33.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926123 bytes = 3125680 } seq expand size = 6482861 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5134 value = 3.4473 dict = 1.00 all = 3.5901 } Zip/UnZip{ index = 0.0951 value = 0.2901 dict = 1.00 all = 0.2785 } ---------------------------- total value len = 0.344364 GB avg = 0.124 KB (by entry num) total key len = 0.077720 GB avg = 0.028 KB total ukey len = 0.055514 GB avg = 0.020 KB total ukey num = 0.002775699 Billion total entry num = 0.002775699 Billion write speed all = 0.008409083 MB/sec (with seq num) write speed all = 0.007966686 MB/sec (without seq num) 2024/11/22-22:48:56.877365 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232190: 926123 keys, 37385982 bytes, temperature: kWarm 2024/11/22-22:48:56.877424 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286936877392, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232190, "file_size": 37385982, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3543969197, "largest_seqno": 3621075430, "table_properties": {"data_size": 31411776, "index_size": 1761792, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931444, "raw_average_key_size": 28, "raw_value_size": 108286414, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926123, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286932, "oldest_key_time": 0, "file_creation_time": 1732286932, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232190, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:48:56.914213 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232191.sst) /dev/shm/ToplingTemp/Topling-321443-2hLEWa 2024/11/22-22:48:57.811958 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232191.sst): first pass time = 0.90's, 149.511'MB/sec 2024/11/22-22:48:59.197696 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232191.sst): old prealloc_size = 147639500, real_size = 32222784 2024/11/22-22:48:59.198287 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232191.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928985 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0302 GB avg-val = 116.48 avg-zval = 32.53 2024/11/22-22:48:59.199061 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232191.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:00.529923 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232191.sst): second pass time = 1.38's, 78.178'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.33's, 27.268'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3097.775'MB/sec (index lex order gen) rebuild zvType time = 1.20's, 0.193'MB/sec write SST data time = 0.13's, 279.747'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 39475.840'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.178'MB/sec zip pipeline throughput = 78.178'MB/sec entries = 928985 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.48 avg-zval = 32.53 usrkeys = 928985 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.48 avg-zval = 32.53 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928985 bytes = 3251456 } seq expand size = 6502895 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0302 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5165 value = 3.5802 dict = 1.00 all = 3.6947 } Zip/UnZip{ index = 0.0951 value = 0.2793 dict = 1.00 all = 0.2707 } ---------------------------- total value len = 0.459073 GB avg = 0.124 KB (by entry num) total key len = 0.103731 GB avg = 0.028 KB total ukey len = 0.074094 GB avg = 0.020 KB total ukey num = 0.003704684 Billion total entry num = 0.003704684 Billion write speed all = 0.011211811 MB/sec (with seq num) write speed all = 0.010621393 MB/sec (without seq num) 2024/11/22-22:49:00.565041 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232191: 928985 keys, 36327118 bytes, temperature: kWarm 2024/11/22-22:49:00.565101 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286940565071, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232191, "file_size": 36327118, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3621075286, "table_properties": {"data_size": 30223664, "index_size": 1766720, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26011580, "raw_average_key_size": 28, "raw_value_size": 108206242, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928985, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286936, "oldest_key_time": 0, "file_creation_time": 1732286936, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232191, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:00.603680 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232192.sst) /dev/shm/ToplingTemp/Topling-321443-AuuJJd 2024/11/22-22:49:01.847956 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232192.sst): first pass time = 1.24's, 107.871'MB/sec 2024/11/22-22:49:03.674746 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232192.sst): old prealloc_size = 147639500, real_size = 33190304 2024/11/22-22:49:03.675291 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232192.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924755 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.14 avg-zval = 33.74 2024/11/22-22:49:03.676063 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232192.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:04.686239 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232192.sst): second pass time = 1.83's, 59.336'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 36.860'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3333.167'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.261'MB/sec write SST data time = 0.13's, 295.678'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33274.401'MB/sec zip my value time = 1.83's, unzip length = 0.108'GB zip my value throughput = 59.336'MB/sec zip pipeline throughput = 117.180'MB/sec entries = 924755 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.14 avg-zval = 33.74 usrkeys = 924755 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.14 avg-zval = 33.74 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924755 bytes = 3236656 } seq expand size = 6473285 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0373 GB } UnZip/Zip{ index = 10.5113 value = 3.4720 dict = 1.00 all = 3.6000 } Zip/UnZip{ index = 0.0951 value = 0.2880 dict = 1.00 all = 0.2778 } ---------------------------- total value len = 0.573871 GB avg = 0.124 KB (by entry num) total key len = 0.129624 GB avg = 0.028 KB total ukey len = 0.092589 GB avg = 0.020 KB total ukey num = 0.004629439 Billion total entry num = 0.004629439 Billion write speed all = 0.014013405 MB/sec (with seq num) write speed all = 0.013275669 MB/sec (without seq num) 2024/11/22-22:49:04.720994 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232192: 924755 keys, 37282604 bytes, temperature: kWarm 2024/11/22-22:49:04.721055 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286944721024, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232192, "file_size": 37282604, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3621074434, "table_properties": {"data_size": 31199376, "index_size": 1759552, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25893140, "raw_average_key_size": 28, "raw_value_size": 108324678, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924755, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286940, "oldest_key_time": 0, "file_creation_time": 1732286940, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232192, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:04.756355 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232193.sst) /dev/shm/ToplingTemp/Topling-321443-pS69fd 2024/11/22-22:49:05.618532 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232193.sst): first pass time = 0.86's, 155.680'MB/sec 2024/11/22-22:49:06.990225 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232193.sst): old prealloc_size = 147639500, real_size = 33174752 2024/11/22-22:49:06.990797 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232193.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924692 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 117.15 avg-zval = 33.72 2024/11/22-22:49:06.991556 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232193.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:08.003727 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232193.sst): second pass time = 1.37's, 79.128'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.01's, 36.657'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3204.892'MB/sec (index lex order gen) rebuild zvType time = 0.88's, 0.261'MB/sec write SST data time = 0.13's, 288.895'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 38090.093'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.128'MB/sec zip pipeline throughput = 79.128'MB/sec entries = 924692 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.15 avg-zval = 33.72 usrkeys = 924692 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.15 avg-zval = 33.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924692 bytes = 3120848 } seq expand size = 6472844 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5136 value = 3.4737 dict = 1.00 all = 3.6127 } Zip/UnZip{ index = 0.0951 value = 0.2879 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 0.688670 GB avg = 0.124 KB (by entry num) total key len = 0.155516 GB avg = 0.028 KB total ukey len = 0.111083 GB avg = 0.020 KB total ukey num = 0.005554131 Billion total entry num = 0.005554131 Billion write speed all = 0.016814808 MB/sec (with seq num) write speed all = 0.015929774 MB/sec (without seq num) 2024/11/22-22:49:08.037257 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232193: 924692 keys, 37151716 bytes, temperature: kWarm 2024/11/22-22:49:08.037322 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286948037290, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232193, "file_size": 37151716, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3522904895, "largest_seqno": 3621074205, "table_properties": {"data_size": 31184336, "index_size": 1759040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25891376, "raw_average_key_size": 28, "raw_value_size": 108326436, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924692, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286944, "oldest_key_time": 0, "file_creation_time": 1732286944, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232193, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:08.072159 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232194.sst) /dev/shm/ToplingTemp/Topling-321443-lvmKkd 2024/11/22-22:49:08.921247 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232194.sst): first pass time = 0.85's, 158.080'MB/sec 2024/11/22-22:49:10.297498 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232194.sst): old prealloc_size = 147639500, real_size = 33551328 2024/11/22-22:49:10.298141 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232194.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925069 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0315 GB avg-val = 117.09 avg-zval = 34.10 2024/11/22-22:49:10.298938 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232194.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:11.360378 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232194.sst): second pass time = 1.38's, 78.739'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 35.418'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2861.094'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.253'MB/sec write SST data time = 0.15's, 253.274'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33360.143'MB/sec zip my value time = 1.38's, unzip length = 0.108'GB zip my value throughput = 78.739'MB/sec zip pipeline throughput = 78.739'MB/sec entries = 925069 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.09 avg-zval = 34.10 usrkeys = 925069 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.09 avg-zval = 34.10 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925069 bytes = 3237760 } seq expand size = 6475483 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4445 value = 3.4333 dict = 1.00 all = 3.5654 } Zip/UnZip{ index = 0.0957 value = 0.2913 dict = 1.00 all = 0.2805 } ---------------------------- total value len = 0.803462 GB avg = 0.124 KB (by entry num) total key len = 0.181418 GB avg = 0.028 KB total ukey len = 0.129584 GB avg = 0.020 KB total ukey num = 0.006479200 Billion total entry num = 0.006479200 Billion write speed all = 0.019615876 MB/sec (with seq num) write speed all = 0.018583505 MB/sec (without seq num) 2024/11/22-22:49:11.396693 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232194: 925069 keys, 37644566 bytes, temperature: kWarm 2024/11/22-22:49:11.396755 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286951396722, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232194, "file_size": 37644566, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3621074220, "table_properties": {"data_size": 31548496, "index_size": 1771392, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25901932, "raw_average_key_size": 28, "raw_value_size": 108315814, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925069, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286948, "oldest_key_time": 0, "file_creation_time": 1732286948, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232194, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:11.435154 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232195.sst) /dev/shm/ToplingTemp/Topling-321443-Ou8YKb 2024/11/22-22:49:12.356267 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232195.sst): first pass time = 0.92's, 145.718'MB/sec 2024/11/22-22:49:13.725091 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232195.sst): old prealloc_size = 147639500, real_size = 33437680 2024/11/22-22:49:13.725639 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232195.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924743 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 117.14 avg-zval = 34.01 2024/11/22-22:49:13.726402 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232195.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:14.760930 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232195.sst): second pass time = 1.37's, 79.177'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.04's, 36.121'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3338.255'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.254'MB/sec write SST data time = 0.13's, 296.871'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 35745.224'MB/sec zip my value time = 1.37's, unzip length = 0.108'GB zip my value throughput = 79.177'MB/sec zip pipeline throughput = 79.177'MB/sec entries = 924743 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.14 avg-zval = 34.01 usrkeys = 924743 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.14 avg-zval = 34.01 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924743 bytes = 3121024 } seq expand size = 6473201 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5119 value = 3.4447 dict = 1.00 all = 3.5873 } Zip/UnZip{ index = 0.0951 value = 0.2903 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 0.918260 GB avg = 0.124 KB (by entry num) total key len = 0.207310 GB avg = 0.028 KB total ukey len = 0.148079 GB avg = 0.020 KB total ukey num = 0.007403943 Billion total entry num = 0.007403943 Billion write speed all = 0.022416506 MB/sec (with seq num) write speed all = 0.021236869 MB/sec (without seq num) 2024/11/22-22:49:14.807850 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232195: 924743 keys, 37414828 bytes, temperature: kWarm 2024/11/22-22:49:14.807910 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286954807880, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232195, "file_size": 37414828, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3499102480, "largest_seqno": 3621074625, "table_properties": {"data_size": 31446880, "index_size": 1759424, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25892804, "raw_average_key_size": 28, "raw_value_size": 108324962, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924743, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286951, "oldest_key_time": 0, "file_creation_time": 1732286951, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232195, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:14.844151 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232196.sst) /dev/shm/ToplingTemp/Topling-321443-Gw2jNd 2024/11/22-22:49:15.744990 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232196.sst): first pass time = 0.90's, 148.998'MB/sec 2024/11/22-22:49:17.187742 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232196.sst): old prealloc_size = 147639500, real_size = 36587584 2024/11/22-22:49:17.188324 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232196.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928614 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0346 GB avg-val = 116.54 avg-zval = 37.24 2024/11/22-22:49:17.188998 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232196.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:17.986249 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232196.sst): second pass time = 1.44's, 75.067'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.80's, 50.109'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3177.085'MB/sec (index lex order gen) rebuild zvType time = 0.67's, 0.348'MB/sec write SST data time = 0.13's, 306.535'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 27138.533'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.067'MB/sec zip pipeline throughput = 90.047'MB/sec entries = 928614 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.54 avg-zval = 37.24 usrkeys = 928614 avg-key = 20.00 avg-zkey = 1.91 avg-val = 121.57 avg-zval = 37.24 TagRS{ kinds = RS_Key0_TagN bytes = 145200 } TagArray{ size = 668031 bytes = 2421632 } seq expand size = 4676217 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0346 GB dict = 1.08 MB all = 0.0400 GB } UnZip/Zip{ index = 10.4442 value = 3.1297 dict = 1.00 all = 3.3545 } Zip/UnZip{ index = 0.0957 value = 0.3195 dict = 1.00 all = 0.2981 } ---------------------------- total value len = 1.031152 GB avg = 0.124 KB (by entry num) total key len = 0.233312 GB avg = 0.028 KB total ukey len = 0.166651 GB avg = 0.020 KB total ukey num = 0.008332557 Billion total entry num = 0.008332557 Billion write speed all = 0.025181058 MB/sec (with seq num) write speed all = 0.023853554 MB/sec (without seq num) 2024/11/22-22:49:18.025790 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232196: 928614 keys, 40011200 bytes, temperature: kWarm 2024/11/22-22:49:18.025850 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286958025818, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232196, "file_size": 40011200, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3621074245, "table_properties": {"data_size": 34577008, "index_size": 1778240, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26001192, "raw_average_key_size": 28, "raw_value_size": 108216564, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928614, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286954, "oldest_key_time": 0, "file_creation_time": 1732286954, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232196, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:18.067210 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232197.sst) /dev/shm/ToplingTemp/Topling-321443-ddnutb 2024/11/22-22:49:18.801329 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232197.sst): first pass time = 0.73's, 182.838'MB/sec 2024/11/22-22:49:20.354592 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232197.sst): old prealloc_size = 147639500, real_size = 36492896 2024/11/22-22:49:20.355146 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232197.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927146 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0345 GB avg-val = 116.76 avg-zval = 37.21 2024/11/22-22:49:20.355190 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232197.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:20.994498 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232197.sst): second pass time = 1.55's, 69.745'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.371'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3299.442'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.450'MB/sec write SST data time = 0.12's, 300.431'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33857.255'MB/sec zip my value time = 1.55's, unzip length = 0.108'GB zip my value throughput = 69.745'MB/sec zip pipeline throughput = 69.745'MB/sec entries = 927146 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 37.21 usrkeys = 927146 avg-key = 20.00 avg-zkey = 1.90 avg-val = 116.76 avg-zval = 37.21 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0345 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5132 value = 3.1382 dict = 1.00 all = 3.5934 } Zip/UnZip{ index = 0.0951 value = 0.3187 dict = 1.00 all = 0.2783 } ---------------------------- total value len = 1.139410 GB avg = 0.123 KB (by entry num) total key len = 0.259272 GB avg = 0.028 KB total ukey len = 0.185194 GB avg = 0.020 KB total ukey num = 0.009259703 Billion total entry num = 0.009259703 Billion write speed all = 0.027852258 MB/sec (with seq num) write speed all = 0.026377134 MB/sec (without seq num) 2024/11/22-22:49:21.028133 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232197: 927146 keys, 37351216 bytes, temperature: kWarm 2024/11/22-22:49:21.028198 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286961028165, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232197, "file_size": 37351216, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34497168, "index_size": 1763776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25960088, "raw_average_key_size": 28, "raw_value_size": 108257744, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927146, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286958, "oldest_key_time": 0, "file_creation_time": 1732286958, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232197, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:21.068090 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232198.sst) /dev/shm/ToplingTemp/Topling-321443-a0Ghld 2024/11/22-22:49:21.853965 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232198.sst): first pass time = 0.79's, 170.797'MB/sec 2024/11/22-22:49:23.414254 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232198.sst): old prealloc_size = 147639500, real_size = 36391200 2024/11/22-22:49:23.414808 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232198.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926880 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0344 GB avg-val = 116.81 avg-zval = 37.12 2024/11/22-22:49:23.414853 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232198.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:24.056708 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232198.sst): second pass time = 1.56's, 69.437'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 57.982'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3290.108'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.448'MB/sec write SST data time = 0.12's, 300.130'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32812.344'MB/sec zip my value time = 1.56's, unzip length = 0.108'GB zip my value throughput = 69.437'MB/sec zip pipeline throughput = 69.437'MB/sec entries = 926880 avg-key = 28.00 avg-zkey = 1.89 avg-val = 116.81 avg-zval = 37.12 usrkeys = 926880 avg-key = 20.00 avg-zkey = 1.89 avg-val = 116.81 avg-zval = 37.12 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 8 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0344 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5828 value = 3.1465 dict = 1.00 all = 3.6032 } Zip/UnZip{ index = 0.0945 value = 0.3178 dict = 1.00 all = 0.2775 } ---------------------------- total value len = 1.247675 GB avg = 0.122 KB (by entry num) total key len = 0.285224 GB avg = 0.028 KB total ukey len = 0.203732 GB avg = 0.020 KB total ukey num = 0.010186583 Billion total entry num = 0.010186583 Billion write speed all = 0.030523106 MB/sec (with seq num) write speed all = 0.028900424 MB/sec (without seq num) 2024/11/22-22:49:24.093030 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232198: 926880 keys, 37249210 bytes, temperature: kWarm 2024/11/22-22:49:24.093092 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286964093060, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232198, "file_size": 37249210, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 34407632, "index_size": 1751680, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25952640, "raw_average_key_size": 28, "raw_value_size": 108265176, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926880, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286961, "oldest_key_time": 0, "file_creation_time": 1732286961, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232198, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:24.129112 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232199.sst) /dev/shm/ToplingTemp/Topling-321443-BRdBsb 2024/11/22-22:49:24.999921 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232199.sst): first pass time = 0.87's, 154.137'MB/sec 2024/11/22-22:49:26.297117 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232199.sst): old prealloc_size = 147639500, real_size = 32016768 2024/11/22-22:49:26.297694 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232199.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 922685 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0300 GB avg-val = 117.46 avg-zval = 32.53 2024/11/22-22:49:26.298461 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232199.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:27.167567 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232199.sst): second pass time = 1.30's, 83.593'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.518'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3189.616'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 275.061'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33899.906'MB/sec zip my value time = 1.30's, unzip length = 0.108'GB zip my value throughput = 83.593'MB/sec zip pipeline throughput = 83.593'MB/sec entries = 922685 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.46 avg-zval = 32.53 usrkeys = 922685 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.99 avg-zval = 32.53 TagRS{ kinds = RS_Key0_TagN bytes = 144280 } TagArray{ size = 859523 bytes = 3115792 } seq expand size = 6016661 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4421 value = 3.6105 dict = 1.00 all = 3.7140 } Zip/UnZip{ index = 0.0958 value = 0.2770 dict = 1.00 all = 0.2693 } ---------------------------- total value len = 1.362075 GB avg = 0.123 KB (by entry num) total key len = 0.311060 GB avg = 0.028 KB total ukey len = 0.222185 GB avg = 0.020 KB total ukey num = 0.011109268 Billion total entry num = 0.011109268 Billion write speed all = 0.033313392 MB/sec (with seq num) write speed all = 0.031543839 MB/sec (without seq num) 2024/11/22-22:49:27.199721 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232199: 922685 keys, 36138670 bytes, temperature: kWarm 2024/11/22-22:49:27.199795 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286967199750, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232199, "file_size": 36138670, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3621074731, "table_properties": {"data_size": 30018672, "index_size": 1767232, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25835180, "raw_average_key_size": 28, "raw_value_size": 108382638, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 922685, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286964, "oldest_key_time": 0, "file_creation_time": 1732286964, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232199, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:27.234551 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232200.sst) /dev/shm/ToplingTemp/Topling-321443-1q8q2e 2024/11/22-22:49:28.102388 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232200.sst): first pass time = 0.87's, 154.666'MB/sec 2024/11/22-22:49:29.435000 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232200.sst): old prealloc_size = 147639500, real_size = 33476144 2024/11/22-22:49:29.435563 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232200.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928204 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0315 GB avg-val = 116.60 avg-zval = 33.91 2024/11/22-22:49:29.436340 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232200.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:30.350168 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232200.sst): second pass time = 1.33's, 81.254'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.935'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3260.817'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.295'MB/sec write SST data time = 0.13's, 290.060'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 34376.027'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.254'MB/sec zip pipeline throughput = 81.254'MB/sec entries = 928204 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.60 avg-zval = 33.91 usrkeys = 928204 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.60 avg-zval = 33.91 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928204 bytes = 3132704 } seq expand size = 6497428 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0375 GB } UnZip/Zip{ index = 10.5118 value = 3.4382 dict = 1.00 all = 3.5828 } Zip/UnZip{ index = 0.0951 value = 0.2908 dict = 1.00 all = 0.2791 } ---------------------------- total value len = 1.476800 GB avg = 0.123 KB (by entry num) total key len = 0.337049 GB avg = 0.028 KB total ukey len = 0.240749 GB avg = 0.020 KB total ukey num = 0.012037472 Billion total entry num = 0.012037472 Billion write speed all = 0.036112853 MB/sec (with seq num) write speed all = 0.034195571 MB/sec (without seq num) 2024/11/22-22:49:30.385491 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232200: 928204 keys, 37462148 bytes, temperature: kWarm 2024/11/22-22:49:30.385557 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286970385523, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232200, "file_size": 37462148, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3509327862, "largest_seqno": 3621074746, "table_properties": {"data_size": 31477920, "index_size": 1766016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25989712, "raw_average_key_size": 28, "raw_value_size": 108228112, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928204, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286967, "oldest_key_time": 0, "file_creation_time": 1732286967, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232200, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:30.422192 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232201.sst) /dev/shm/ToplingTemp/Topling-321443-1NeMad 2024/11/22-22:49:31.267859 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232201.sst): first pass time = 0.85's, 158.721'MB/sec 2024/11/22-22:49:32.609796 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232201.sst): old prealloc_size = 147639500, real_size = 33935568 2024/11/22-22:49:32.610365 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232201.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 934761 prefix = 4 raw-key = 0.0150 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1080 GB zip-val = 0.0319 GB avg-val = 115.59 avg-zval = 34.14 2024/11/22-22:49:32.611190 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232201.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:33.509874 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232201.sst): second pass time = 1.34's, 80.591'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.413'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3258.313'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.303'MB/sec write SST data time = 0.13's, 299.065'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 25655.466'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.591'MB/sec zip pipeline throughput = 80.591'MB/sec entries = 934761 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.59 avg-zval = 34.14 usrkeys = 934761 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.59 avg-zval = 34.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 934761 bytes = 3388528 } seq expand size = 6543327 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0187 GB value = 0.1080 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0382 GB } UnZip/Zip{ index = 10.4442 value = 3.3857 dict = 1.00 all = 3.5159 } Zip/UnZip{ index = 0.0957 value = 0.2954 dict = 1.00 all = 0.2844 } ---------------------------- total value len = 1.591388 GB avg = 0.123 KB (by entry num) total key len = 0.363223 GB avg = 0.028 KB total ukey len = 0.259445 GB avg = 0.020 KB total ukey num = 0.012972233 Billion total entry num = 0.012972233 Billion write speed all = 0.038912891 MB/sec (with seq num) write speed all = 0.036846854 MB/sec (without seq num) 2024/11/22-22:49:33.544601 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232201: 934761 keys, 38174430 bytes, temperature: kWarm 2024/11/22-22:49:33.544662 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286973544631, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232201, "file_size": 38174430, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701386, "largest_seqno": 3621075320, "table_properties": {"data_size": 31911680, "index_size": 1790016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26173308, "raw_average_key_size": 28, "raw_value_size": 108044534, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 934761, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286970, "oldest_key_time": 0, "file_creation_time": 1732286970, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232201, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:33.593685 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232202.sst) /dev/shm/ToplingTemp/Topling-321443-A6cOIc 2024/11/22-22:49:34.510326 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232202.sst): first pass time = 0.92's, 146.429'MB/sec 2024/11/22-22:49:35.819754 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232202.sst): old prealloc_size = 147639500, real_size = 31703040 2024/11/22-22:49:35.820303 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232202.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924421 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0297 GB avg-val = 117.19 avg-zval = 32.14 2024/11/22-22:49:35.821661 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232202.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-22:49:36.681179 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232202.sst): second pass time = 1.31's, 82.776'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 41.554'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3414.825'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.315'MB/sec write SST data time = 0.13's, 283.110'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.082'MB, throughput = 29962.603'MB/sec zip my value time = 1.31's, unzip length = 0.108'GB zip my value throughput = 82.776'MB/sec zip pipeline throughput = 82.776'MB/sec entries = 924421 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.19 avg-zval = 32.14 usrkeys = 924421 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.19 avg-zval = 32.14 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924421 bytes = 3235488 } seq expand size = 6470947 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5140 value = 3.6460 dict = 1.00 all = 3.7496 } Zip/UnZip{ index = 0.0951 value = 0.2743 dict = 1.00 all = 0.2667 } ---------------------------- total value len = 1.706193 GB avg = 0.123 KB (by entry num) total key len = 0.389106 GB avg = 0.028 KB total ukey len = 0.277933 GB avg = 0.020 KB total ukey num = 0.013896654 Billion total entry num = 0.013896654 Billion write speed all = 0.041711126 MB/sec (with seq num) write speed all = 0.039498000 MB/sec (without seq num) 2024/11/22-22:49:36.720299 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232202: 924421 keys, 35794852 bytes, temperature: kWarm 2024/11/22-22:49:36.720365 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286976720330, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232202, "file_size": 35794852, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3470221528, "largest_seqno": 3621073121, "table_properties": {"data_size": 29713264, "index_size": 1758464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25883788, "raw_average_key_size": 28, "raw_value_size": 108334054, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924421, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286973, "oldest_key_time": 0, "file_creation_time": 1732286973, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232202, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:36.755141 140024928073280 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232203.sst) /dev/shm/ToplingTemp/Topling-321443-ficEdf 2024/11/22-22:49:36.839702 140024928073280 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232203.sst): first pass time = 0.08's, 239.126'MB/sec 2024/11/22-22:49:37.026833 140024928073280 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232203.sst): old prealloc_size = 147639500, real_size = 5909376 2024/11/22-22:49:37.026962 140024928073280 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232203.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 140890 prefix = 4 raw-key = 0.0023 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.0163 GB zip-val = 0.0056 GB avg-val = 115.45 avg-zval = 39.78 2024/11/22-22:49:37.027108 140024928073280 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232203.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.020 GB 2024/11/22-22:49:37.157305 140024928073280 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00/232203.sst): second pass time = 0.19's, 87.202'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.13's, 50.132'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2550.903'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.338'MB/sec write SST data time = 0.03's, 250.673'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.171'MB, throughput = 4103.015'MB/sec zip my value time = 0.19's, unzip length = 0.016'GB zip my value throughput = 87.202'MB/sec zip pipeline throughput = 87.202'MB/sec entries = 140890 avg-key = 28.00 avg-zkey = 1.91 avg-val = 115.45 avg-zval = 39.78 usrkeys = 140890 avg-key = 20.00 avg-zkey = 1.91 avg-val = 122.45 avg-zval = 39.78 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 140890 bytes = 493136 } seq expand size = 986230 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0028 GB value = 0.0163 GB dict = 0.17 MB all = 0.0202 GB } __ZipSize{ index = 0.0003 GB value = 0.0056 GB dict = 0.17 MB all = 0.0065 GB } UnZip/Zip{ index = 10.4679 value = 2.9022 dict = 1.00 all = 3.0903 } Zip/UnZip{ index = 0.0955 value = 0.3446 dict = 1.00 all = 0.3236 } ---------------------------- total value len = 1.723446 GB avg = 0.123 KB (by entry num) total key len = 0.393051 GB avg = 0.028 KB total ukey len = 0.280751 GB avg = 0.020 KB total ukey num = 0.014037544 Billion total entry num = 0.014037544 Billion write speed all = 0.042132705 MB/sec (with seq num) write speed all = 0.039897163 MB/sec (without seq num) 2024/11/22-22:49:37.170094 140024928073280 [compaction_job.cc:1948] [default] [JOB 13174] Generated table #232203: 140890 keys, 6540179 bytes, temperature: kWarm 2024/11/22-22:49:37.170149 140024928073280 EVENT_LOG_v1 {"time_micros": 1732286977170119, "cf_name": "default", "job": 13174, "event": "table_file_creation", "file_number": 232203, "file_size": 6540179, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3619567031, "table_properties": {"data_size": 5604784, "index_size": 269184, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3944920, "raw_average_key_size": 28, "raw_value_size": 16266356, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 140890, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732286976, "oldest_key_time": 0, "file_creation_time": 1732286976, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 232203, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-22:49:37.180398 140024928073280 [compaction_job.cc:703] job-13174: subcompact[0], size: 0.566475 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-13174/att-00 [232188,232189,232190,232191,232192,232193,232194,232195,232196,232197,232198,232199,232200,232201,232202,232203] 2024/11/22-22:49:37.282041 140024928073280 (Original Log Time 2024/11/22-22:48:41.319404) EVENT_LOG_v1 {"time_micros": 1732286921319349, "job": 13174, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [232182], "files_L3": [232109, 232110, 232111, 232112, 232113, 232114, 232115, 232116, 232117, 232118, 232119, 232120, 232121, 232122, 232123], "score": 0.998705, "input_data_size": 587847271, "oldest_snapshot_seqno": 3621161099} 2024/11/22-22:49:37.282044 140024928073280 (Original Log Time 2024/11/22-22:49:37.281227) [compaction_job.cc:2022] [default] [JOB 13174] Compacted 1@2 + 15@3 files to L3 => 566474541 bytes 2024/11/22-22:49:37.282046 140024928073280 (Original Log Time 2024/11/22-22:49:37.281979) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 16 0 0 0] max score 0.02, MB/sec: 10.5 rd, 10.1 wr, level 3, files in(1, 15) out(16 +0 blob) MB in(27.1, 533.5 +0.0 blob) out(540.2 +0.0 blob), read-write-amplify(40.6) write-amplify(19.9) OK, records in: 14131236, records dropped: 93692 output_compression: Snappy 2024/11/22-22:49:37.282051 140024928073280 (Original Log Time 2024/11/22-22:49:37.282022) EVENT_LOG_v1 {"time_micros": 1732286977282002, "cf": "default", "job": 13174, "event": "compaction_finished", "compaction_time_micros": 55860999, "compaction_time_cpu_micros": 24091994, "output_level": 3, "num_output_files": 16, "total_output_size": 566474541, "num_input_records": 14131236, "num_output_records": 14037544, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 364812326, "file_range_sync_nanos": 0, "file_fsync_nanos": 13830, "file_prepare_write_nanos": 36552, "lsm_state": [0, 0, 0, 16, 0, 0, 0]} 2024/11/22-22:49:37.421565 140024928073280 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732286921,"executesMs":56088,"compactionJobId":13174,"attempt":0,"compactionInputRawBytes":2046782748,"compactionInputZipBytes":587847271,"compactionOutputRawBytes":0,"compactionOutputZipBytes":566474541,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-d6337460cd1c19b69b9fed7cd54bb2b2-6bddf5be6fa741ec-00"}}, response = 2024/11/22-22:49:37.421634 140024928073280 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-13174/att-00: olev 3, work 56.088 s, result 6.364 ms, install 0.843 ms, input{raw 1.906 GiB zip 560.615 MiB}