2024/11/23-06:42:38.339289 140025163068992 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00: fork to child time = 0.009212 sec 2024/11/23-06:42:38.340150 140025163068992 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00 2024/11/23-06:42:38.347893 140025163068992 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00 2024/11/23-06:42:38.349042 140025163068992 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-06:42:38.349329 140025163068992 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-06:42:38.406149 140025163068992 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21233/att-00/MANIFEST-206347 2024/11/23-06:42:38.438159 140025163068992 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21233/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 249038, last_sequence is 3841900820, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 248888 2024/11/23-06:42:38.438172 140025163068992 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-06:42:38.438406 140025163068992 [version_set.cc:5721] Creating manifest 249038 2024/11/23-06:42:38.469719 140025163068992 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-06:42:38.488933 140025163068992 [compaction_job.cc:2437] [default] [JOB 21233] Compacting 1@2 + 14@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-06:42:38.488957 140025163068992 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 15900 Base level 2, inputs: [249035(27MB)], [248988(33MB) 248989(34MB) 248990(35MB) 248991(35MB) 248992(35MB) 248993(33MB) 248994(34MB) 248995(36MB) 248996(36MB) 248997(36MB) 248998(35MB) 248999(34MB) 249000(34MB) 249001(35MB)] 2024/11/23-06:42:38.497479 140025163068992 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:42:38.497522 140025163068992 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:42:38.497550 140025163068992 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-06:42:38.497559 140025163068992 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:42:38.497573 140025163068992 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-06:42:38.961037 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249039.sst) /dev/shm/ToplingTemp/Topling-437650-7ORaqe 2024/11/23-06:42:39.543850 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249039.sst): first pass time = 0.58's, 230.309'MB/sec 2024/11/23-06:42:41.773900 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249039.sst): old prealloc_size = 147639500, real_size = 31962000 2024/11/23-06:42:41.774459 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249039.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923425 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1084 GB zip-val = 0.0300 GB avg-val = 117.35 avg-zval = 32.45 2024/11/23-06:42:41.775278 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249039.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:42:42.900254 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249039.sst): second pass time = 2.23's, 48.606'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.13's, 32.008'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3268.793'MB/sec (index lex order gen) rebuild zvType time = 1.00's, 0.232'MB/sec write SST data time = 0.13's, 277.046'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 32600.187'MB/sec zip my value time = 2.23's, unzip length = 0.108'GB zip my value throughput = 48.606'MB/sec zip pipeline throughput = 73.963'MB/sec entries = 923425 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.35 avg-zval = 32.45 usrkeys = 923425 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.35 avg-zval = 32.45 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923425 bytes = 3232000 } seq expand size = 6463975 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0300 GB dict = 1.08 MB all = 0.0361 GB } UnZip/Zip{ index = 10.4430 value = 3.6166 dict = 1.00 all = 3.7229 } Zip/UnZip{ index = 0.0958 value = 0.2765 dict = 1.00 all = 0.2686 } ---------------------------- total value len = 0.114826 GB avg = 0.124 KB (by entry num) total key len = 0.025856 GB avg = 0.028 KB total ukey len = 0.018468 GB avg = 0.020 KB total ukey num = 0.000923425 Billion total entry num = 0.000923425 Billion write speed all = 0.001789393 MB/sec (with seq num) write speed all = 0.001695430 MB/sec (without seq num) 2024/11/23-06:42:42.934005 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249039: 923425 keys, 36051854 bytes, temperature: kWarm 2024/11/23-06:42:42.934094 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315362934041, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249039, "file_size": 36051854, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3661838825, "largest_seqno": 3841879707, "table_properties": {"data_size": 29962432, "index_size": 1768512, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25855900, "raw_average_key_size": 28, "raw_value_size": 108361882, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315358, "oldest_key_time": 0, "file_creation_time": 1732315358, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249039, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:42:42.973316 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249040.sst) /dev/shm/ToplingTemp/Topling-437650-0y1yjf 2024/11/23-06:42:44.095914 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249040.sst): first pass time = 1.12's, 119.564'MB/sec 2024/11/23-06:42:46.121977 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249040.sst): old prealloc_size = 147639500, real_size = 31936480 2024/11/23-06:42:46.122501 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249040.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932217 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0299 GB avg-val = 115.98 avg-zval = 32.11 2024/11/23-06:42:46.123267 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249040.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:42:47.234163 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249040.sst): second pass time = 2.03's, 53.387'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.11's, 32.309'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3379.125'MB/sec (index lex order gen) rebuild zvType time = 0.91's, 0.256'MB/sec write SST data time = 0.20's, 179.888'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30165.622'MB/sec zip my value time = 2.03's, unzip length = 0.108'GB zip my value throughput = 53.387'MB/sec zip pipeline throughput = 84.053'MB/sec entries = 932217 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.98 avg-zval = 32.11 usrkeys = 932217 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.98 avg-zval = 32.11 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932217 bytes = 3146240 } seq expand size = 6525519 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0299 GB dict = 1.08 MB all = 0.0359 GB } UnZip/Zip{ index = 10.5112 value = 3.6123 dict = 1.00 all = 3.7352 } Zip/UnZip{ index = 0.0951 value = 0.2768 dict = 1.00 all = 0.2677 } ---------------------------- total value len = 0.229467 GB avg = 0.124 KB (by entry num) total key len = 0.051958 GB avg = 0.028 KB total ukey len = 0.037113 GB avg = 0.020 KB total ukey num = 0.001855642 Billion total entry num = 0.001855642 Billion write speed all = 0.003579373 MB/sec (with seq num) write speed all = 0.003390561 MB/sec (without seq num) 2024/11/23-06:42:47.267359 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249040: 932217 keys, 35933382 bytes, temperature: kWarm 2024/11/23-06:42:47.267417 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315367267387, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249040, "file_size": 35933382, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3748831343, "largest_seqno": 3841879068, "table_properties": {"data_size": 29929488, "index_size": 1773760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26102076, "raw_average_key_size": 28, "raw_value_size": 108115726, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932217, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315362, "oldest_key_time": 0, "file_creation_time": 1732315362, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249040, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:42:47.307209 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249041.sst) /dev/shm/ToplingTemp/Topling-437650-LlSZUf 2024/11/23-06:42:48.420826 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249041.sst): first pass time = 1.11's, 120.529'MB/sec 2024/11/23-06:42:50.422313 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249041.sst): old prealloc_size = 147639500, real_size = 33661776 2024/11/23-06:42:50.422959 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249041.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929769 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0317 GB avg-val = 116.36 avg-zval = 34.05 2024/11/23-06:42:50.425231 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249041.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:42:51.560719 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249041.sst): second pass time = 2.00's, 54.072'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.14's, 33.279'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2826.705'MB/sec (index lex order gen) rebuild zvType time = 0.98's, 0.238'MB/sec write SST data time = 0.16's, 235.501'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 37020.722'MB/sec zip my value time = 2.00's, unzip length = 0.108'GB zip my value throughput = 54.072'MB/sec zip pipeline throughput = 54.072'MB/sec entries = 929769 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.36 avg-zval = 34.05 usrkeys = 929769 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.36 avg-zval = 34.05 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929769 bytes = 3370432 } seq expand size = 6508383 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5132 value = 3.4170 dict = 1.00 all = 3.5428 } Zip/UnZip{ index = 0.0951 value = 0.2927 dict = 1.00 all = 0.2823 } ---------------------------- total value len = 0.344160 GB avg = 0.124 KB (by entry num) total key len = 0.077992 GB avg = 0.028 KB total ukey len = 0.055708 GB avg = 0.020 KB total ukey num = 0.002785411 Billion total entry num = 0.002785411 Billion write speed all = 0.005368937 MB/sec (with seq num) write speed all = 0.005085538 MB/sec (without seq num) 2024/11/23-06:42:51.592970 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249041: 929769 keys, 37884664 bytes, temperature: kWarm 2024/11/23-06:42:51.593033 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315371593002, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249041, "file_size": 37884664, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038075, "largest_seqno": 3841879759, "table_properties": {"data_size": 31660352, "index_size": 1768768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26033532, "raw_average_key_size": 28, "raw_value_size": 108184302, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929769, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315367, "oldest_key_time": 0, "file_creation_time": 1732315367, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249041, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:42:51.637155 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249042.sst) /dev/shm/ToplingTemp/Topling-437650-8KePhf 2024/11/23-06:42:52.577644 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249042.sst): first pass time = 0.94's, 142.717'MB/sec 2024/11/23-06:42:53.939060 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249042.sst): old prealloc_size = 147639500, real_size = 32789728 2024/11/23-06:42:53.939619 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249042.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 924394 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0308 GB avg-val = 117.20 avg-zval = 33.31 2024/11/23-06:42:53.940444 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249042.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:42:54.864252 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249042.sst): second pass time = 1.36's, 79.615'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.989'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3298.906'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.290'MB/sec write SST data time = 0.13's, 288.657'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 32992.067'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.615'MB/sec zip pipeline throughput = 79.615'MB/sec entries = 924394 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.20 avg-zval = 33.31 usrkeys = 924394 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.20 avg-zval = 33.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 924394 bytes = 3350944 } seq expand size = 6470758 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0308 GB dict = 1.08 MB all = 0.0370 GB } UnZip/Zip{ index = 10.4448 value = 3.5187 dict = 1.00 all = 3.6278 } Zip/UnZip{ index = 0.0957 value = 0.2842 dict = 1.00 all = 0.2756 } ---------------------------- total value len = 0.458965 GB avg = 0.124 KB (by entry num) total key len = 0.103875 GB avg = 0.028 KB total ukey len = 0.074196 GB avg = 0.020 KB total ukey num = 0.003709805 Billion total entry num = 0.003709805 Billion write speed all = 0.007157920 MB/sec (with seq num) write speed all = 0.006780484 MB/sec (without seq num) 2024/11/23-06:42:54.896940 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249042: 924394 keys, 36996756 bytes, temperature: kWarm 2024/11/23-06:42:54.896998 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315374896967, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249042, "file_size": 36996756, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038106, "largest_seqno": 3841879833, "table_properties": {"data_size": 30788368, "index_size": 1770048, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25883032, "raw_average_key_size": 28, "raw_value_size": 108334788, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 924394, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315371, "oldest_key_time": 0, "file_creation_time": 1732315371, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249042, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:42:54.937263 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249043.sst) /dev/shm/ToplingTemp/Topling-437650-LfyIIe 2024/11/23-06:42:55.800094 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249043.sst): first pass time = 0.86's, 155.562'MB/sec 2024/11/23-06:42:57.379683 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249043.sst): old prealloc_size = 147639500, real_size = 32488544 2024/11/23-06:42:57.380232 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249043.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932173 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0305 GB avg-val = 115.98 avg-zval = 32.70 2024/11/23-06:42:57.381008 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249043.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:42:58.325520 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249043.sst): second pass time = 1.58's, 68.473'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.452'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3357.738'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.288'MB/sec write SST data time = 0.14's, 269.320'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 28801.496'MB/sec zip my value time = 1.58's, unzip length = 0.108'GB zip my value throughput = 68.473'MB/sec zip pipeline throughput = 128.143'MB/sec entries = 932173 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.98 avg-zval = 32.70 usrkeys = 932173 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.98 avg-zval = 32.70 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932173 bytes = 3029584 } seq expand size = 6525211 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5134 value = 3.5469 dict = 1.00 all = 3.6904 } Zip/UnZip{ index = 0.0951 value = 0.2819 dict = 1.00 all = 0.2710 } ---------------------------- total value len = 0.573608 GB avg = 0.124 KB (by entry num) total key len = 0.129975 GB avg = 0.028 KB total ukey len = 0.092840 GB avg = 0.020 KB total ukey num = 0.004641978 Billion total entry num = 0.004641978 Billion write speed all = 0.008947427 MB/sec (with seq num) write speed all = 0.008475172 MB/sec (without seq num) 2024/11/23-06:42:58.357632 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249043: 932173 keys, 36369080 bytes, temperature: kWarm 2024/11/23-06:42:58.357695 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315378357665, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249043, "file_size": 36369080, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3781358041, "largest_seqno": 3841878960, "table_properties": {"data_size": 30482000, "index_size": 1773312, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26100844, "raw_average_key_size": 28, "raw_value_size": 108117010, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932173, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315374, "oldest_key_time": 0, "file_creation_time": 1732315374, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249043, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:42:58.392338 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249044.sst) /dev/shm/ToplingTemp/Topling-437650-GZnypf 2024/11/23-06:42:59.188864 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249044.sst): first pass time = 0.80's, 168.515'MB/sec 2024/11/23-06:43:00.778562 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249044.sst): old prealloc_size = 147639500, real_size = 31449952 2024/11/23-06:43:00.779101 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249044.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 920261 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1085 GB zip-val = 0.0295 GB avg-val = 117.85 avg-zval = 32.02 2024/11/23-06:43:00.780727 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249044.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:43:01.644642 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249044.sst): second pass time = 1.59's, 68.251'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.158'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3383.795'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.313'MB/sec write SST data time = 0.13's, 273.819'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.084'MB, throughput = 29091.875'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 68.251'MB/sec zip pipeline throughput = 136.331'MB/sec entries = 920261 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.85 avg-zval = 32.02 usrkeys = 920261 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.85 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 920261 bytes = 3335968 } seq expand size = 6441827 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0295 GB dict = 1.08 MB all = 0.0356 GB } UnZip/Zip{ index = 10.5145 value = 3.6801 dict = 1.00 all = 3.7653 } Zip/UnZip{ index = 0.0951 value = 0.2717 dict = 1.00 all = 0.2656 } ---------------------------- total value len = 0.688500 GB avg = 0.124 KB (by entry num) total key len = 0.155743 GB avg = 0.028 KB total ukey len = 0.111245 GB avg = 0.020 KB total ukey num = 0.005562239 Billion total entry num = 0.005562239 Billion write speed all = 0.010735735 MB/sec (with seq num) write speed all = 0.010169881 MB/sec (without seq num) 2024/11/23-06:43:01.692630 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249044: 920261 keys, 35645876 bytes, temperature: kWarm 2024/11/23-06:43:01.692693 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315381692662, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249044, "file_size": 35645876, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3434038136, "largest_seqno": 3841879263, "table_properties": {"data_size": 29469264, "index_size": 1750464, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25767308, "raw_average_key_size": 28, "raw_value_size": 108450542, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 920261, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315378, "oldest_key_time": 0, "file_creation_time": 1732315378, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249044, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:43:01.732590 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249045.sst) /dev/shm/ToplingTemp/Topling-437650-gQIjdf 2024/11/23-06:43:02.541740 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249045.sst): first pass time = 0.81's, 165.883'MB/sec 2024/11/23-06:43:04.169857 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249045.sst): old prealloc_size = 147639500, real_size = 32425136 2024/11/23-06:43:04.170432 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249045.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 932870 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1081 GB zip-val = 0.0304 GB avg-val = 115.88 avg-zval = 32.61 2024/11/23-06:43:04.172245 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249045.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:43:05.122284 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249045.sst): second pass time = 1.63's, 66.440'MB/sec, value only(80.5% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 38.244'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3199.752'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.285'MB/sec write SST data time = 0.13's, 275.165'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.078'MB, throughput = 30407.686'MB/sec zip my value time = 1.63's, unzip length = 0.108'GB zip my value throughput = 66.440'MB/sec zip pipeline throughput = 132.450'MB/sec entries = 932870 avg-key = 28.00 avg-zkey = 1.90 avg-val = 115.88 avg-zval = 32.61 usrkeys = 932870 avg-key = 20.00 avg-zkey = 1.90 avg-val = 122.88 avg-zval = 32.61 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 932870 bytes = 3148448 } seq expand size = 6530090 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0187 GB value = 0.1081 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0304 GB dict = 1.08 MB all = 0.0364 GB } UnZip/Zip{ index = 10.5121 value = 3.5539 dict = 1.00 all = 3.6849 } Zip/UnZip{ index = 0.0951 value = 0.2814 dict = 1.00 all = 0.2714 } ---------------------------- total value len = 0.803128 GB avg = 0.124 KB (by entry num) total key len = 0.181863 GB avg = 0.028 KB total ukey len = 0.129902 GB avg = 0.020 KB total ukey num = 0.006495109 Billion total entry num = 0.006495109 Billion write speed all = 0.012524990 MB/sec (with seq num) write speed all = 0.011864263 MB/sec (without seq num) 2024/11/23-06:43:05.156822 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249045: 932870 keys, 36423810 bytes, temperature: kWarm 2024/11/23-06:43:05.156899 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315385156850, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249045, "file_size": 36423810, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3753310698, "largest_seqno": 3841880058, "table_properties": {"data_size": 30416864, "index_size": 1774848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26120360, "raw_average_key_size": 28, "raw_value_size": 108097496, "raw_average_value_size": 115, "num_data_blocks": 1, "num_entries": 932870, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315381, "oldest_key_time": 0, "file_creation_time": 1732315381, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249045, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:43:05.192444 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249046.sst) /dev/shm/ToplingTemp/Topling-437650-UPCY3g 2024/11/23-06:43:06.013975 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249046.sst): first pass time = 0.82's, 163.383'MB/sec 2024/11/23-06:43:07.671948 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249046.sst): old prealloc_size = 147639500, real_size = 36731920 2024/11/23-06:43:07.672504 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249046.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929234 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1082 GB zip-val = 0.0347 GB avg-val = 116.44 avg-zval = 37.35 2024/11/23-06:43:07.673303 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249046.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:43:08.351240 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249046.sst): second pass time = 1.66's, 65.308'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.68's, 56.533'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3334.384'MB/sec (index lex order gen) rebuild zvType time = 0.55's, 0.423'MB/sec write SST data time = 0.13's, 296.721'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31942.811'MB/sec zip my value time = 1.66's, unzip length = 0.108'GB zip my value throughput = 65.308'MB/sec zip pipeline throughput = 72.467'MB/sec entries = 929234 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.44 avg-zval = 37.35 usrkeys = 929234 avg-key = 20.00 avg-zkey = 1.93 avg-val = 117.78 avg-zval = 37.35 TagRS{ kinds = RS_Key0_TagN bytes = 145280 } TagArray{ size = 177621 bytes = 666096 } seq expand size = 1243347 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0347 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.3757 value = 3.1174 dict = 1.00 all = 3.4953 } Zip/UnZip{ index = 0.0964 value = 0.3208 dict = 1.00 all = 0.2861 } ---------------------------- total value len = 0.912570 GB avg = 0.123 KB (by entry num) total key len = 0.207882 GB avg = 0.028 KB total ukey len = 0.148487 GB avg = 0.020 KB total ukey num = 0.007424343 Billion total entry num = 0.007424343 Billion write speed all = 0.014246907 MB/sec (with seq num) write speed all = 0.013491684 MB/sec (without seq num) 2024/11/23-06:43:08.396347 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249046: 929234 keys, 38399651 bytes, temperature: kWarm 2024/11/23-06:43:08.396401 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315388396373, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249046, "file_size": 38399651, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3841879537, "table_properties": {"data_size": 34708288, "index_size": 1791168, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26018552, "raw_average_key_size": 28, "raw_value_size": 108199196, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929234, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315385, "oldest_key_time": 0, "file_creation_time": 1732315385, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249046, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:43:08.430377 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249047.sst) /dev/shm/ToplingTemp/Topling-437650-iZTVof 2024/11/23-06:43:09.189417 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249047.sst): first pass time = 0.76's, 176.835'MB/sec 2024/11/23-06:43:10.879233 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249047.sst): old prealloc_size = 147639500, real_size = 36754800 2024/11/23-06:43:10.879756 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249047.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926695 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0348 GB avg-val = 116.83 avg-zval = 37.51 2024/11/23-06:43:10.879923 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249047.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:43:11.522762 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249047.sst): second pass time = 1.69's, 64.099'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.64's, 58.902'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3517.707'MB/sec (index lex order gen) rebuild zvType time = 0.52's, 0.447'MB/sec write SST data time = 0.12's, 303.449'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 36103.631'MB/sec zip my value time = 1.69's, unzip length = 0.108'GB zip my value throughput = 64.099'MB/sec zip pipeline throughput = 64.099'MB/sec entries = 926695 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.83 avg-zval = 37.51 usrkeys = 926695 avg-key = 20.00 avg-zkey = 1.90 avg-val = 117.20 avg-zval = 37.51 TagRS{ kinds = RS_Key0_TagN bytes = 144880 } TagArray{ size = 48817 bytes = 146464 } seq expand size = 341719 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0348 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.5127 value = 3.1148 dict = 1.00 all = 3.5409 } Zip/UnZip{ index = 0.0951 value = 0.3210 dict = 1.00 all = 0.2824 } ---------------------------- total value len = 1.021182 GB avg = 0.122 KB (by entry num) total key len = 0.233829 GB avg = 0.028 KB total ukey len = 0.167021 GB avg = 0.020 KB total ukey num = 0.008351038 Billion total entry num = 0.008351038 Billion write speed all = 0.015957232 MB/sec (with seq num) write speed all = 0.015107777 MB/sec (without seq num) 2024/11/23-06:43:11.573098 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249047: 926695 keys, 37904641 bytes, temperature: kWarm 2024/11/23-06:43:11.573160 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315391573128, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249047, "file_size": 37904641, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3283714205, "table_properties": {"data_size": 34759904, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25947460, "raw_average_key_size": 28, "raw_value_size": 108270378, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926695, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315388, "oldest_key_time": 0, "file_creation_time": 1732315388, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249047, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:43:11.610159 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249048.sst) /dev/shm/ToplingTemp/Topling-437650-YGSK9c 2024/11/23-06:43:12.444285 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249048.sst): first pass time = 0.83's, 160.917'MB/sec 2024/11/23-06:43:14.031120 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249048.sst): old prealloc_size = 147639500, real_size = 33570752 2024/11/23-06:43:14.031662 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249048.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925935 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0316 GB avg-val = 116.95 avg-zval = 34.09 2024/11/23-06:43:14.032526 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249048.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:43:14.981444 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249048.sst): second pass time = 1.59's, 68.272'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.879'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3496.020'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.283'MB/sec write SST data time = 0.13's, 286.650'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33501.782'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 68.272'MB/sec zip pipeline throughput = 134.281'MB/sec entries = 925935 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.95 avg-zval = 34.09 usrkeys = 925935 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.95 avg-zval = 34.09 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925935 bytes = 3472272 } seq expand size = 6481545 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0379 GB } UnZip/Zip{ index = 10.4434 value = 3.4307 dict = 1.00 all = 3.5415 } Zip/UnZip{ index = 0.0958 value = 0.2915 dict = 1.00 all = 0.2824 } ---------------------------- total value len = 1.135955 GB avg = 0.122 KB (by entry num) total key len = 0.259755 GB avg = 0.028 KB total ukey len = 0.185539 GB avg = 0.020 KB total ukey num = 0.009276973 Billion total entry num = 0.009276973 Billion write speed all = 0.017745418 MB/sec (with seq num) write speed all = 0.016801820 MB/sec (without seq num) 2024/11/23-06:43:15.014690 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249048: 925935 keys, 37898094 bytes, temperature: kWarm 2024/11/23-06:43:15.014750 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315395014718, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249048, "file_size": 37898094, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3283714206, "largest_seqno": 3841880106, "table_properties": {"data_size": 31565808, "index_size": 1773248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25926180, "raw_average_key_size": 28, "raw_value_size": 108291682, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 925935, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315391, "oldest_key_time": 0, "file_creation_time": 1732315391, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249048, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:43:15.053209 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249049.sst) /dev/shm/ToplingTemp/Topling-437650-PPms2f 2024/11/23-06:43:15.912374 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249049.sst): first pass time = 0.86's, 156.226'MB/sec 2024/11/23-06:43:17.450423 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249049.sst): old prealloc_size = 147639500, real_size = 33311136 2024/11/23-06:43:17.450960 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249049.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928425 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0313 GB avg-val = 116.57 avg-zval = 33.73 2024/11/23-06:43:17.451761 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249049.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:43:18.403850 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249049.sst): second pass time = 1.54's, 70.397'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.242'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3407.681'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.282'MB/sec write SST data time = 0.13's, 290.933'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 25417.938'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.397'MB/sec zip pipeline throughput = 115.839'MB/sec entries = 928425 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.57 avg-zval = 33.73 usrkeys = 928425 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.57 avg-zval = 33.73 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928425 bytes = 3249504 } seq expand size = 6498975 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5113 value = 3.4562 dict = 1.00 all = 3.5874 } Zip/UnZip{ index = 0.0951 value = 0.2893 dict = 1.00 all = 0.2788 } ---------------------------- total value len = 1.250676 GB avg = 0.123 KB (by entry num) total key len = 0.285751 GB avg = 0.028 KB total ukey len = 0.204108 GB avg = 0.020 KB total ukey num = 0.010205398 Billion total entry num = 0.010205398 Billion write speed all = 0.019533677 MB/sec (with seq num) write speed all = 0.018495690 MB/sec (without seq num) 2024/11/23-06:43:18.439552 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249049: 928425 keys, 37414106 bytes, temperature: kWarm 2024/11/23-06:43:18.439615 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315398439584, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249049, "file_size": 37414106, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3631595348, "largest_seqno": 3841879857, "table_properties": {"data_size": 31312336, "index_size": 1766528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25995900, "raw_average_key_size": 28, "raw_value_size": 108221870, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315395, "oldest_key_time": 0, "file_creation_time": 1732315395, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249049, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:43:18.474370 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249050.sst) /dev/shm/ToplingTemp/Topling-437650-zxjvNd 2024/11/23-06:43:19.345619 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249050.sst): first pass time = 0.87's, 154.059'MB/sec 2024/11/23-06:43:20.748305 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249050.sst): old prealloc_size = 147639500, real_size = 30730560 2024/11/23-06:43:20.748846 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249050.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927920 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0287 GB avg-val = 116.64 avg-zval = 30.95 2024/11/23-06:43:20.749622 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249050.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:43:21.627345 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249050.sst): second pass time = 1.40's, 77.201'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 39.492'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3439.078'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.311'MB/sec write SST data time = 0.13's, 261.173'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33478.487'MB/sec zip my value time = 1.40's, unzip length = 0.108'GB zip my value throughput = 77.201'MB/sec zip pipeline throughput = 84.992'MB/sec entries = 927920 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.64 avg-zval = 30.95 usrkeys = 927920 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.64 avg-zval = 30.95 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927920 bytes = 3131744 } seq expand size = 6495440 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0287 GB dict = 1.08 MB all = 0.0347 GB } UnZip/Zip{ index = 10.4443 value = 3.7685 dict = 1.00 all = 3.8663 } Zip/UnZip{ index = 0.0957 value = 0.2654 dict = 1.00 all = 0.2586 } ---------------------------- total value len = 1.365408 GB avg = 0.123 KB (by entry num) total key len = 0.311733 GB avg = 0.028 KB total ukey len = 0.222666 GB avg = 0.020 KB total ukey num = 0.011133318 Billion total entry num = 0.011133318 Billion write speed all = 0.021321788 MB/sec (with seq num) write speed all = 0.020189469 MB/sec (without seq num) 2024/11/23-06:43:21.657744 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249050: 927920 keys, 34714336 bytes, temperature: kWarm 2024/11/23-06:43:21.657811 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315401657774, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249050, "file_size": 34714336, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3718597675, "largest_seqno": 3841879877, "table_properties": {"data_size": 28721520, "index_size": 1776896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25981760, "raw_average_key_size": 28, "raw_value_size": 108235968, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927920, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315398, "oldest_key_time": 0, "file_creation_time": 1732315398, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249050, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:43:21.693130 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249051.sst) /dev/shm/ToplingTemp/Topling-437650-Wfajgf 2024/11/23-06:43:22.501510 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249051.sst): first pass time = 0.81's, 166.042'MB/sec 2024/11/23-06:43:24.045450 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249051.sst): old prealloc_size = 147639500, real_size = 31207760 2024/11/23-06:43:24.045984 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249051.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 929348 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1082 GB zip-val = 0.0292 GB avg-val = 116.42 avg-zval = 31.42 2024/11/23-06:43:24.046776 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249051.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:43:24.922054 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249051.sst): second pass time = 1.54's, 70.108'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 40.285'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3465.385'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.312'MB/sec write SST data time = 0.13's, 269.709'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 33159.237'MB/sec zip my value time = 1.54's, unzip length = 0.108'GB zip my value throughput = 70.108'MB/sec zip pipeline throughput = 121.534'MB/sec entries = 929348 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.42 avg-zval = 31.42 usrkeys = 929348 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.42 avg-zval = 31.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 929348 bytes = 3252736 } seq expand size = 6505436 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0292 GB dict = 1.08 MB all = 0.0353 GB } UnZip/Zip{ index = 10.4449 value = 3.7059 dict = 1.00 all = 3.8008 } Zip/UnZip{ index = 0.0957 value = 0.2698 dict = 1.00 all = 0.2631 } ---------------------------- total value len = 1.480109 GB avg = 0.123 KB (by entry num) total key len = 0.337755 GB avg = 0.028 KB total ukey len = 0.241253 GB avg = 0.020 KB total ukey num = 0.012062666 Billion total entry num = 0.012062666 Billion write speed all = 0.023109860 MB/sec (with seq num) write speed all = 0.021883073 MB/sec (without seq num) 2024/11/23-06:43:24.975357 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249051: 929348 keys, 35313364 bytes, temperature: kWarm 2024/11/23-06:43:24.975409 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315404975381, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249051, "file_size": 35313364, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3583362754, "largest_seqno": 3841879897, "table_properties": {"data_size": 29195712, "index_size": 1779520, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26021744, "raw_average_key_size": 28, "raw_value_size": 108196088, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 929348, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315401, "oldest_key_time": 0, "file_creation_time": 1732315401, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249051, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:43:25.029022 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249052.sst) /dev/shm/ToplingTemp/Topling-437650-L6CiOc 2024/11/23-06:43:25.826804 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249052.sst): first pass time = 0.80's, 168.247'MB/sec 2024/11/23-06:43:27.420432 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249052.sst): old prealloc_size = 147639500, real_size = 31738208 2024/11/23-06:43:27.420989 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249052.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928720 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0297 GB avg-val = 116.52 avg-zval = 32.02 2024/11/23-06:43:27.421768 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249052.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-06:43:28.310233 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249052.sst): second pass time = 1.59's, 67.934'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.89's, 40.282'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3292.936'MB/sec (index lex order gen) rebuild zvType time = 0.76's, 0.305'MB/sec write SST data time = 0.13's, 278.052'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 28579.475'MB/sec zip my value time = 1.59's, unzip length = 0.108'GB zip my value throughput = 67.934'MB/sec zip pipeline throughput = 134.310'MB/sec entries = 928720 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.52 avg-zval = 32.02 usrkeys = 928720 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.52 avg-zval = 32.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928720 bytes = 3250528 } seq expand size = 6501040 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.08 MB all = 0.0358 GB } UnZip/Zip{ index = 10.5127 value = 3.6388 dict = 1.00 all = 3.7447 } Zip/UnZip{ index = 0.0951 value = 0.2748 dict = 1.00 all = 0.2670 } ---------------------------- total value len = 1.594824 GB avg = 0.123 KB (by entry num) total key len = 0.363759 GB avg = 0.028 KB total ukey len = 0.259828 GB avg = 0.020 KB total ukey num = 0.012991386 Billion total entry num = 0.012991386 Billion write speed all = 0.024897698 MB/sec (with seq num) write speed all = 0.023576515 MB/sec (without seq num) 2024/11/23-06:43:28.342811 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249052: 928720 keys, 35842088 bytes, temperature: kWarm 2024/11/23-06:43:28.342874 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315408342842, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249052, "file_size": 35842088, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3590663973, "largest_seqno": 3841879908, "table_properties": {"data_size": 29739024, "index_size": 1766848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26004160, "raw_average_key_size": 28, "raw_value_size": 108213712, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928720, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315405, "oldest_key_time": 0, "file_creation_time": 1732315405, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249052, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:43:28.383550 140025163068992 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249053.sst) /dev/shm/ToplingTemp/Topling-437650-cZSFye 2024/11/23-06:43:28.480769 140025163068992 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249053.sst): first pass time = 0.10's, 235.949'MB/sec 2024/11/23-06:43:28.697235 140025163068992 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249053.sst): old prealloc_size = 147639500, real_size = 6688640 2024/11/23-06:43:28.697370 140025163068992 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249053.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 157045 prefix = 4 raw-key = 0.0025 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0185 GB zip-val = 0.0063 GB avg-val = 118.00 avg-zval = 40.40 2024/11/23-06:43:28.697520 140025163068992 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249053.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0004 GB, waited 0.000 sec, Key+Value = 0.023 GB 2024/11/23-06:43:28.846381 140025163068992 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00/249053.sst): second pass time = 0.22's, 85.817'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.15's, 49.188'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2743.584'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.310'MB/sec write SST data time = 0.02's, 328.998'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.194'MB, throughput = 6848.493'MB/sec zip my value time = 0.22's, unzip length = 0.019'GB zip my value throughput = 85.817'MB/sec zip pipeline throughput = 85.817'MB/sec entries = 157045 avg-key = 28.00 avg-zkey = 1.93 avg-val = 118.00 avg-zval = 40.40 usrkeys = 157045 avg-key = 20.00 avg-zkey = 1.93 avg-val = 125.00 avg-zval = 40.40 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 157045 bytes = 490784 } seq expand size = 1099315 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0031 GB value = 0.0185 GB dict = 0.19 MB all = 0.0229 GB } __ZipSize{ index = 0.0003 GB value = 0.0063 GB dict = 0.19 MB all = 0.0073 GB } UnZip/Zip{ index = 10.3384 value = 2.9204 dict = 1.00 all = 3.1256 } Zip/UnZip{ index = 0.0967 value = 0.3424 dict = 1.00 all = 0.3199 } ---------------------------- total value len = 1.614455 GB avg = 0.123 KB (by entry num) total key len = 0.368156 GB avg = 0.028 KB total ukey len = 0.262969 GB avg = 0.020 KB total ukey num = 0.013148431 Billion total entry num = 0.013148431 Billion write speed all = 0.025202971 MB/sec (with seq num) write speed all = 0.023865827 MB/sec (without seq num) 2024/11/23-06:43:28.857315 140025163068992 [compaction_job.cc:1948] [default] [JOB 21233] Generated table #249053: 157045 keys, 7335729 bytes, temperature: kWarm 2024/11/23-06:43:28.857373 140025163068992 EVENT_LOG_v1 {"time_micros": 1732315408857343, "cf_name": "default", "job": 21233, "event": "table_file_creation", "file_number": 249053, "file_size": 7335729, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3783761883, "largest_seqno": 3815184243, "table_properties": {"data_size": 6345392, "index_size": 303808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 4397260, "raw_average_key_size": 28, "raw_value_size": 18531310, "raw_average_value_size": 118, "num_data_blocks": 1, "num_entries": 157045, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 118, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732315408, "oldest_key_time": 0, "file_creation_time": 1732315408, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 249053, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-06:43:28.872051 140025163068992 [compaction_job.cc:703] job-21233: subcompact[0], size: 0.520127 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-21233/att-00 [249039,249040,249041,249042,249043,249044,249045,249046,249047,249048,249049,249050,249051,249052,249053] 2024/11/23-06:43:28.966411 140025163068992 (Original Log Time 2024/11/23-06:42:38.489033) EVENT_LOG_v1 {"time_micros": 1732315358488977, "job": 21233, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [249035], "files_L3": [248988, 248989, 248990, 248991, 248992, 248993, 248994, 248995, 248996, 248997, 248998, 248999, 249000, 249001], "score": 0.99341, "input_data_size": 543155618, "oldest_snapshot_seqno": 3841900670} 2024/11/23-06:43:28.966414 140025163068992 (Original Log Time 2024/11/23-06:43:28.964869) [compaction_job.cc:2022] [default] [JOB 21233] Compacted 1@2 + 14@3 files to L3 => 520127431 bytes 2024/11/23-06:43:28.966416 140025163068992 (Original Log Time 2024/11/23-06:43:28.966342) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 15 0 0 0] max score 0.02, MB/sec: 10.8 rd, 10.3 wr, level 3, files in(1, 14) out(15 +0 blob) MB in(27.1, 490.9 +0.0 blob) out(496.0 +0.0 blob), read-write-amplify(37.4) write-amplify(18.3) OK, records in: 13223282, records dropped: 74851 output_compression: Snappy 2024/11/23-06:43:28.966425 140025163068992 (Original Log Time 2024/11/23-06:43:28.966391) EVENT_LOG_v1 {"time_micros": 1732315408966367, "cf": "default", "job": 21233, "event": "compaction_finished", "compaction_time_micros": 50383023, "compaction_time_cpu_micros": 21350062, "output_level": 3, "num_output_files": 15, "total_output_size": 520127431, "num_input_records": 13223282, "num_output_records": 13148431, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 348560081, "file_range_sync_nanos": 0, "file_fsync_nanos": 12823, "file_prepare_write_nanos": 29793, "lsm_state": [0, 0, 0, 15, 0, 0, 0]} 2024/11/23-06:43:29.085525 140025163068992 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732315358,"executesMs":50620,"compactionJobId":21233,"attempt":0,"compactionInputRawBytes":1912606860,"compactionInputZipBytes":543155618,"compactionOutputRawBytes":0,"compactionOutputZipBytes":520127431,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-2e36fe3e502d0795f90edf3f7a0308d9-85f84aab48ef3429-00"}}, response = 2024/11/23-06:43:29.085593 140025163068992 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-21233/att-00: olev 3, work 50.621 s, result 4.827 ms, install 1.578 ms, input{raw 1.781 GiB zip 517.994 MiB}