2024/11/23-00:42:50.800088 140024751826496 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00: fork to child time = 0.008937 sec 2024/11/23-00:42:50.801008 140024751826496 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00 2024/11/23-00:42:50.808402 140024751826496 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00 2024/11/23-00:42:50.809531 140024751826496 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-00:42:50.809836 140024751826496 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-00:42:50.855377 140024751826496 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15523/att-00/MANIFEST-206347 2024/11/23-00:42:50.868642 140024751826496 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15523/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 236938, last_sequence is 3682991406, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 236720 2024/11/23-00:42:50.868650 140024751826496 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-00:42:50.868761 140024751826496 [version_set.cc:5721] Creating manifest 236938 2024/11/23-00:42:50.901580 140024751826496 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-00:42:50.920219 140024751826496 [compaction_job.cc:2437] [default] [JOB 15523] Compacting 1@2 + 13@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-00:42:50.920238 140024751826496 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 11632 Base level 2, inputs: [236878(21MB)], [236935(35MB) 236936(6994KB) 236832(33MB) 236833(34MB) 236834(35MB) 236835(35MB) 236836(35MB) 236837(33MB) 236838(35MB) 236839(34MB) 236840(18MB) 236793(35MB) 236913(34MB)] 2024/11/23-00:42:50.925455 140024751826496 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:42:50.925488 140024751826496 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:42:50.925509 140024751826496 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-00:42:50.925514 140024751826496 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-00:42:50.925526 140024751826496 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-00:42:51.455557 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236939.sst) /dev/shm/ToplingTemp/Topling-354810-VNy74g 2024/11/23-00:42:52.004878 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236939.sst): first pass time = 0.55's, 244.355'MB/sec 2024/11/23-00:42:53.348106 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236939.sst): old prealloc_size = 147639500, real_size = 33194672 2024/11/23-00:42:53.348656 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236939.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926938 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0312 GB avg-val = 116.80 avg-zval = 33.66 2024/11/23-00:42:53.349438 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236939.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:42:54.278881 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236939.sst): second pass time = 1.34's, 80.646'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.942'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3341.018'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 293.645'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33767.721'MB/sec zip my value time = 1.34's, unzip length = 0.108'GB zip my value throughput = 80.646'MB/sec zip pipeline throughput = 80.646'MB/sec entries = 926938 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.80 avg-zval = 33.66 usrkeys = 926938 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.80 avg-zval = 33.66 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926938 bytes = 3128432 } seq expand size = 6488566 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0312 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5150 value = 3.4700 dict = 1.00 all = 3.6102 } Zip/UnZip{ index = 0.0951 value = 0.2882 dict = 1.00 all = 0.2770 } ---------------------------- total value len = 0.114752 GB avg = 0.124 KB (by entry num) total key len = 0.025954 GB avg = 0.028 KB total ukey len = 0.018539 GB avg = 0.020 KB total ukey num = 0.000926938 Billion total entry num = 0.000926938 Billion write speed all = 0.002467182 MB/sec (with seq num) write speed all = 0.002337157 MB/sec (without seq num) 2024/11/23-00:42:54.311141 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236939: 926938 keys, 37177664 bytes, temperature: kWarm 2024/11/23-00:42:54.311232 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293774311176, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236939, "file_size": 37177664, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3613197573, "largest_seqno": 3682495636, "table_properties": {"data_size": 31199712, "index_size": 1763072, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25954264, "raw_average_key_size": 28, "raw_value_size": 108263476, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926938, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293771, "oldest_key_time": 0, "file_creation_time": 1732293771, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236939, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:42:54.347172 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236940.sst) /dev/shm/ToplingTemp/Topling-354810-m63Hbf 2024/11/23-00:42:55.218455 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236940.sst): first pass time = 0.87's, 154.054'MB/sec 2024/11/23-00:42:56.538374 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236940.sst): old prealloc_size = 147639500, real_size = 32729280 2024/11/23-00:42:56.538960 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236940.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927053 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.1083 GB zip-val = 0.0307 GB avg-val = 116.78 avg-zval = 33.13 2024/11/23-00:42:56.539727 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236940.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:42:57.470509 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236940.sst): second pass time = 1.32's, 82.083'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.93's, 39.634'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3170.207'MB/sec (index lex order gen) rebuild zvType time = 0.80's, 0.288'MB/sec write SST data time = 0.13's, 290.075'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 33481.773'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 82.083'MB/sec zip pipeline throughput = 82.083'MB/sec entries = 927053 avg-key = 28.00 avg-zkey = 1.93 avg-val = 116.78 avg-zval = 33.13 usrkeys = 927053 avg-key = 20.00 avg-zkey = 1.93 avg-val = 123.78 avg-zval = 33.13 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927053 bytes = 3360576 } seq expand size = 6489371 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0307 GB dict = 1.08 MB all = 0.0369 GB } UnZip/Zip{ index = 10.3777 value = 3.5252 dict = 1.00 all = 3.6330 } Zip/UnZip{ index = 0.0964 value = 0.2837 dict = 1.00 all = 0.2753 } ---------------------------- total value len = 0.229502 GB avg = 0.124 KB (by entry num) total key len = 0.051912 GB avg = 0.028 KB total ukey len = 0.037080 GB avg = 0.020 KB total ukey num = 0.001853991 Billion total entry num = 0.001853991 Billion write speed all = 0.004934103 MB/sec (with seq num) write speed all = 0.004674050 MB/sec (without seq num) 2024/11/23-00:42:57.502874 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236940: 927053 keys, 36943946 bytes, temperature: kWarm 2024/11/23-00:42:57.502939 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293777502908, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236940, "file_size": 36943946, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3376774746, "largest_seqno": 3682495780, "table_properties": {"data_size": 30710704, "index_size": 1786624, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25957484, "raw_average_key_size": 28, "raw_value_size": 108260290, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927053, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293774, "oldest_key_time": 0, "file_creation_time": 1732293774, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236940, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:42:57.542873 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236941.sst) /dev/shm/ToplingTemp/Topling-354810-0dCCxf 2024/11/23-00:42:58.433112 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236941.sst): first pass time = 0.89's, 150.774'MB/sec 2024/11/23-00:42:59.756704 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236941.sst): old prealloc_size = 147639500, real_size = 32335712 2024/11/23-00:42:59.757260 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236941.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928682 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0303 GB avg-val = 116.52 avg-zval = 32.67 2024/11/23-00:42:59.758011 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236941.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:43:00.677141 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236941.sst): second pass time = 1.32's, 81.803'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 39.464'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3302.442'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.293'MB/sec write SST data time = 0.13's, 286.456'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 35614.240'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.803'MB/sec zip pipeline throughput = 81.803'MB/sec entries = 928682 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.52 avg-zval = 32.67 usrkeys = 928682 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.52 avg-zval = 32.67 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928682 bytes = 3134320 } seq expand size = 6500774 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0303 GB dict = 1.08 MB all = 0.0363 GB } UnZip/Zip{ index = 10.5138 value = 3.5671 dict = 1.00 all = 3.6951 } Zip/UnZip{ index = 0.0951 value = 0.2803 dict = 1.00 all = 0.2706 } ---------------------------- total value len = 0.344217 GB avg = 0.124 KB (by entry num) total key len = 0.077915 GB avg = 0.028 KB total ukey len = 0.055653 GB avg = 0.020 KB total ukey num = 0.002782673 Billion total entry num = 0.002782673 Billion write speed all = 0.007400944 MB/sec (with seq num) write speed all = 0.007010651 MB/sec (without seq num) 2024/11/23-00:43:00.709349 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236941: 928682 keys, 36323096 bytes, temperature: kWarm 2024/11/23-00:43:00.709413 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293780709381, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236941, "file_size": 36323096, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3563652023, "largest_seqno": 3682495660, "table_properties": {"data_size": 30336784, "index_size": 1766592, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26003096, "raw_average_key_size": 28, "raw_value_size": 108214644, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928682, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293777, "oldest_key_time": 0, "file_creation_time": 1732293777, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236941, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:43:00.743509 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236942.sst) /dev/shm/ToplingTemp/Topling-354810-TsOzwe 2024/11/23-00:43:01.621187 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236942.sst): first pass time = 0.88's, 152.931'MB/sec 2024/11/23-00:43:02.942473 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236942.sst): old prealloc_size = 147639500, real_size = 32592432 2024/11/23-00:43:02.943063 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236942.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928716 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0306 GB avg-val = 116.52 avg-zval = 32.94 2024/11/23-00:43:02.943794 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236942.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:43:03.840163 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236942.sst): second pass time = 1.32's, 81.944'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 40.879'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3118.872'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.301'MB/sec write SST data time = 0.13's, 289.330'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.079'MB, throughput = 31965.058'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.944'MB/sec zip pipeline throughput = 81.944'MB/sec entries = 928716 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.52 avg-zval = 32.94 usrkeys = 928716 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.52 avg-zval = 32.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 928716 bytes = 3250528 } seq expand size = 6501012 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5173 value = 3.5371 dict = 1.00 all = 3.6575 } Zip/UnZip{ index = 0.0951 value = 0.2827 dict = 1.00 all = 0.2734 } ---------------------------- total value len = 0.458932 GB avg = 0.124 KB (by entry num) total key len = 0.103919 GB avg = 0.028 KB total ukey len = 0.074228 GB avg = 0.020 KB total ukey num = 0.003711389 Billion total entry num = 0.003711389 Billion write speed all = 0.009867521 MB/sec (with seq num) write speed all = 0.009346997 MB/sec (without seq num) 2024/11/23-00:43:03.873255 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236942: 928716 keys, 36696116 bytes, temperature: kWarm 2024/11/23-00:43:03.873319 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293783873285, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236942, "file_size": 36696116, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3504428200, "largest_seqno": 3682496000, "table_properties": {"data_size": 30594016, "index_size": 1766080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26004048, "raw_average_key_size": 28, "raw_value_size": 108213740, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928716, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293780, "oldest_key_time": 0, "file_creation_time": 1732293780, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236942, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:43:03.907643 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236943.sst) /dev/shm/ToplingTemp/Topling-354810-TCFY0e 2024/11/23-00:43:04.710786 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236943.sst): first pass time = 0.80's, 167.126'MB/sec 2024/11/23-00:43:06.065588 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236943.sst): old prealloc_size = 147639500, real_size = 33496464 2024/11/23-00:43:06.066180 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236943.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 928654 prefix = 4 raw-key = 0.0149 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0315 GB avg-val = 116.53 avg-zval = 33.92 2024/11/23-00:43:06.066747 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236943.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:43:06.926273 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236943.sst): second pass time = 1.35's, 79.917'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.86's, 42.594'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3095.790'MB/sec (index lex order gen) rebuild zvType time = 0.73's, 0.318'MB/sec write SST data time = 0.13's, 284.362'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 31612.658'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 79.917'MB/sec zip pipeline throughput = 79.917'MB/sec entries = 928654 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.53 avg-zval = 33.92 usrkeys = 928654 avg-key = 20.00 avg-zkey = 1.90 avg-val = 121.35 avg-zval = 33.92 TagRS{ kinds = RS_Key0_TagN bytes = 145200 } TagArray{ size = 640152 bytes = 2160528 } seq expand size = 4481064 multi value expand size = 0 cnt WriteAppend = 16 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0315 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5154 value = 3.4356 dict = 1.00 all = 3.6612 } Zip/UnZip{ index = 0.0951 value = 0.2911 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 0.571628 GB avg = 0.123 KB (by entry num) total key len = 0.129921 GB avg = 0.028 KB total ukey len = 0.092801 GB avg = 0.020 KB total ukey num = 0.004640043 Billion total entry num = 0.004640043 Billion write speed all = 0.012298432 MB/sec (with seq num) write speed all = 0.011647698 MB/sec (without seq num) 2024/11/23-00:43:06.960750 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236943: 928654 keys, 36659648 bytes, temperature: kWarm 2024/11/23-00:43:06.960822 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293786960790, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236943, "file_size": 36659648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3682495928, "table_properties": {"data_size": 31497856, "index_size": 1766272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 26002312, "raw_average_key_size": 28, "raw_value_size": 108215512, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 928654, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293783, "oldest_key_time": 0, "file_creation_time": 1732293783, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236943, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:43:06.997188 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236944.sst) /dev/shm/ToplingTemp/Topling-354810-TDpfGi 2024/11/23-00:43:07.915578 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236944.sst): first pass time = 0.92's, 146.152'MB/sec 2024/11/23-00:43:09.313937 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236944.sst): old prealloc_size = 147639500, real_size = 34953696 2024/11/23-00:43:09.314597 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236944.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 1081415 prefix = 4 raw-key = 0.0173 GB zip-key = 0.0021 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1039 GB zip-val = 0.0326 GB avg-val = 96.11 avg-zval = 30.16 2024/11/23-00:43:09.314646 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236944.sst: sumWaitingMem = 0.003 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0028 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:43:09.937763 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236944.sst): second pass time = 1.40's, 74.369'MB/sec, value only(77.4% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.62's, 57.274'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3256.068'MB/sec (index lex order gen) rebuild zvType time = 0.49's, 0.557'MB/sec write SST data time = 0.14's, 258.653'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.034'MB, throughput = 26760.071'MB/sec zip my value time = 1.40's, unzip length = 0.104'GB zip my value throughput = 74.369'MB/sec zip pipeline throughput = 74.369'MB/sec entries = 1081415 avg-key = 28.00 avg-zkey = 1.91 avg-val = 96.11 avg-zval = 30.16 usrkeys = 1081415 avg-key = 20.00 avg-zkey = 1.91 avg-val = 96.11 avg-zval = 30.16 TagRS{ kinds = LegacyZvType bytes = 0 } TagArray{ size = 0 bytes = 0 } seq expand size = 0 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0216 GB value = 0.1039 GB dict = 1.03 MB all = 0.1342 GB } __ZipSize{ index = 0.0021 GB value = 0.0326 GB dict = 1.03 MB all = 0.0357 GB } UnZip/Zip{ index = 10.4484 value = 3.1870 dict = 1.00 all = 3.7566 } Zip/UnZip{ index = 0.0957 value = 0.3138 dict = 1.00 all = 0.2662 } ---------------------------- total value len = 0.675567 GB avg = 0.118 KB (by entry num) total key len = 0.160201 GB avg = 0.028 KB total ukey len = 0.114429 GB avg = 0.020 KB total ukey num = 0.005721458 Billion total entry num = 0.005721458 Billion write speed all = 0.014650546 MB/sec (with seq num) write speed all = 0.013848194 MB/sec (without seq num) 2024/11/23-00:43:09.971136 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236944: 1081415 keys, 35728460 bytes, temperature: kWarm 2024/11/23-00:43:09.971205 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293789971171, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236944, "file_size": 35728460, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 0, "table_properties": {"data_size": 32613136, "index_size": 2070016, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 30279620, "raw_average_key_size": 28, "raw_value_size": 103938115, "raw_average_value_size": 96, "num_data_blocks": 1, "num_entries": 1081415, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "TooZip", "compression_options": "LegacyZvType-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293786, "oldest_key_time": 0, "file_creation_time": 1732293786, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236944, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:43:10.005210 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236945.sst) /dev/shm/ToplingTemp/Topling-354810-ZbMOYf 2024/11/23-00:43:10.905658 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236945.sst): first pass time = 0.90's, 149.064'MB/sec 2024/11/23-00:43:12.229294 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236945.sst): old prealloc_size = 147639500, real_size = 31718560 2024/11/23-00:43:12.229900 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236945.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 919540 prefix = 4 raw-key = 0.0147 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1085 GB zip-val = 0.0297 GB avg-val = 117.96 avg-zval = 32.33 2024/11/23-00:43:12.230676 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236945.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:43:13.098291 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236945.sst): second pass time = 1.32's, 81.995'MB/sec, value only(80.8% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.87's, 41.420'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3004.170'MB/sec (index lex order gen) rebuild zvType time = 0.74's, 0.312'MB/sec write SST data time = 0.13's, 274.682'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.085'MB, throughput = 34413.524'MB/sec zip my value time = 1.32's, unzip length = 0.108'GB zip my value throughput = 81.995'MB/sec zip pipeline throughput = 81.995'MB/sec entries = 919540 avg-key = 28.00 avg-zkey = 1.91 avg-val = 117.96 avg-zval = 32.33 usrkeys = 919540 avg-key = 20.00 avg-zkey = 1.91 avg-val = 124.82 avg-zval = 32.33 TagRS{ kinds = RS_Key0_TagN bytes = 143760 } TagArray{ size = 901054 bytes = 3266336 } seq expand size = 6307378 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0184 GB value = 0.1085 GB dict = 1.09 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0297 GB dict = 1.09 MB all = 0.0360 GB } UnZip/Zip{ index = 10.4448 value = 3.6488 dict = 1.00 all = 3.7290 } Zip/UnZip{ index = 0.0957 value = 0.2741 dict = 1.00 all = 0.2682 } ---------------------------- total value len = 0.790345 GB avg = 0.119 KB (by entry num) total key len = 0.185948 GB avg = 0.028 KB total ukey len = 0.132820 GB avg = 0.020 KB total ukey num = 0.006640998 Billion total entry num = 0.006640998 Billion write speed all = 0.017112928 MB/sec (with seq num) write speed all = 0.016181675 MB/sec (without seq num) 2024/11/23-00:43:13.132667 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236945: 919540 keys, 35993200 bytes, temperature: kWarm 2024/11/23-00:43:13.132774 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293793132701, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236945, "file_size": 35993200, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3606155228, "table_properties": {"data_size": 29727696, "index_size": 1760768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25747120, "raw_average_key_size": 28, "raw_value_size": 108470696, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 919540, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293789, "oldest_key_time": 0, "file_creation_time": 1732293789, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236945, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:43:13.166377 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236946.sst) /dev/shm/ToplingTemp/Topling-354810-i2KPle 2024/11/23-00:43:13.952233 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236946.sst): first pass time = 0.79's, 170.802'MB/sec 2024/11/23-00:43:15.300467 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236946.sst): old prealloc_size = 147639500, real_size = 33058928 2024/11/23-00:43:15.301014 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236946.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926111 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0311 GB avg-val = 116.93 avg-zval = 33.54 2024/11/23-00:43:15.301749 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236946.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:43:16.219615 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236946.sst): second pass time = 1.35's, 80.377'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 40.424'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3375.421'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.292'MB/sec write SST data time = 0.13's, 294.547'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 34963.699'MB/sec zip my value time = 1.35's, unzip length = 0.108'GB zip my value throughput = 80.377'MB/sec zip pipeline throughput = 80.377'MB/sec entries = 926111 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.93 avg-zval = 33.54 usrkeys = 926111 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.93 avg-zval = 33.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926111 bytes = 3241408 } seq expand size = 6482777 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0311 GB dict = 1.08 MB all = 0.0372 GB } UnZip/Zip{ index = 10.5102 value = 3.4858 dict = 1.00 all = 3.6123 } Zip/UnZip{ index = 0.0951 value = 0.2869 dict = 1.00 all = 0.2768 } ---------------------------- total value len = 0.905114 GB avg = 0.120 KB (by entry num) total key len = 0.211879 GB avg = 0.028 KB total ukey len = 0.151342 GB avg = 0.020 KB total ukey num = 0.007567109 Billion total entry num = 0.007567109 Billion write speed all = 0.019578125 MB/sec (with seq num) write speed all = 0.018517063 MB/sec (without seq num) 2024/11/23-00:43:16.251226 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236946: 926111 keys, 37155362 bytes, temperature: kWarm 2024/11/23-00:43:16.251290 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293796251257, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236946, "file_size": 37155362, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3487497056, "largest_seqno": 3682496083, "table_properties": {"data_size": 31064928, "index_size": 1762304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931108, "raw_average_key_size": 28, "raw_value_size": 108286698, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926111, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293793, "oldest_key_time": 0, "file_creation_time": 1732293793, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236946, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:43:16.285923 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236947.sst) /dev/shm/ToplingTemp/Topling-354810-VUIJni 2024/11/23-00:43:17.097606 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236947.sst): first pass time = 0.81's, 165.367'MB/sec 2024/11/23-00:43:18.431658 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236947.sst): old prealloc_size = 147639500, real_size = 33355376 2024/11/23-00:43:18.432250 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236947.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926170 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.92 avg-zval = 33.85 2024/11/23-00:43:18.433581 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236947.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:43:19.384532 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236947.sst): second pass time = 1.33's, 81.214'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.426'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3110.756'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.281'MB/sec write SST data time = 0.13's, 294.209'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 32992.242'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.214'MB/sec zip pipeline throughput = 81.214'MB/sec entries = 926170 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.92 avg-zval = 33.85 usrkeys = 926170 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.92 avg-zval = 33.85 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926170 bytes = 3357376 } seq expand size = 6483190 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4438 value = 3.4541 dict = 1.00 all = 3.5727 } Zip/UnZip{ index = 0.0958 value = 0.2895 dict = 1.00 all = 0.2799 } ---------------------------- total value len = 1.019882 GB avg = 0.120 KB (by entry num) total key len = 0.237812 GB avg = 0.028 KB total ukey len = 0.169866 GB avg = 0.020 KB total ukey num = 0.008493279 Billion total entry num = 0.008493279 Billion write speed all = 0.022043042 MB/sec (with seq num) write speed all = 0.020852179 MB/sec (without seq num) 2024/11/23-00:43:19.419911 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236947: 926170 keys, 37567286 bytes, temperature: kWarm 2024/11/23-00:43:19.419978 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293799419944, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236947, "file_size": 37567286, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3375334738, "largest_seqno": 3682496095, "table_properties": {"data_size": 31350048, "index_size": 1773632, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25932760, "raw_average_key_size": 28, "raw_value_size": 108285064, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926170, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293796, "oldest_key_time": 0, "file_creation_time": 1732293796, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236947, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:43:19.453097 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236948.sst) /dev/shm/ToplingTemp/Topling-354810-YjgFff 2024/11/23-00:43:20.550466 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236948.sst): first pass time = 1.10's, 122.313'MB/sec 2024/11/23-00:43:21.879440 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236948.sst): old prealloc_size = 147639500, real_size = 35043264 2024/11/23-00:43:21.880002 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236948.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927014 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0330 GB avg-val = 116.79 avg-zval = 35.64 2024/11/23-00:43:21.881359 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236948.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:43:22.784535 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236948.sst): second pass time = 1.33's, 81.513'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 43.375'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3304.514'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.298'MB/sec write SST data time = 0.13's, 307.073'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34363.469'MB/sec zip my value time = 1.33's, unzip length = 0.108'GB zip my value throughput = 81.513'MB/sec zip pipeline throughput = 81.513'MB/sec entries = 927014 avg-key = 28.00 avg-zkey = 1.92 avg-val = 116.79 avg-zval = 35.64 usrkeys = 927014 avg-key = 20.00 avg-zkey = 1.92 avg-val = 123.79 avg-zval = 35.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 927014 bytes = 3360448 } seq expand size = 6489098 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0330 GB dict = 1.08 MB all = 0.0393 GB } UnZip/Zip{ index = 10.4431 value = 3.2771 dict = 1.00 all = 3.4189 } Zip/UnZip{ index = 0.0958 value = 0.3051 dict = 1.00 all = 0.2925 } ---------------------------- total value len = 1.134633 GB avg = 0.120 KB (by entry num) total key len = 0.263768 GB avg = 0.028 KB total ukey len = 0.188406 GB avg = 0.020 KB total ukey num = 0.009420293 Billion total entry num = 0.009420293 Billion write speed all = 0.024507689 MB/sec (with seq num) write speed all = 0.023186926 MB/sec (without seq num) 2024/11/23-00:43:22.819283 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236948: 927014 keys, 39258058 bytes, temperature: kWarm 2024/11/23-00:43:22.819348 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293802819315, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236948, "file_size": 39258058, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701103, "largest_seqno": 3682496134, "table_properties": {"data_size": 33035952, "index_size": 1775360, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25956392, "raw_average_key_size": 28, "raw_value_size": 108261424, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927014, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293799, "oldest_key_time": 0, "file_creation_time": 1732293799, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236948, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:43:22.857002 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236949.sst) /dev/shm/ToplingTemp/Topling-354810-nEqeHg 2024/11/23-00:43:23.755173 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236949.sst): first pass time = 0.90's, 149.442'MB/sec 2024/11/23-00:43:25.120159 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236949.sst): old prealloc_size = 147639500, real_size = 33427840 2024/11/23-00:43:25.120750 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236949.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926124 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0314 GB avg-val = 116.92 avg-zval = 33.93 2024/11/23-00:43:25.122101 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236949.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-00:43:26.072191 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236949.sst): second pass time = 1.36's, 79.371'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.95's, 39.537'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3120.372'MB/sec (index lex order gen) rebuild zvType time = 0.82's, 0.281'MB/sec write SST data time = 0.13's, 294.511'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 34758.090'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.371'MB/sec zip pipeline throughput = 79.371'MB/sec entries = 926124 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.92 avg-zval = 33.93 usrkeys = 926124 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.92 avg-zval = 33.93 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926124 bytes = 3357216 } seq expand size = 6482868 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0314 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.4463 value = 3.4461 dict = 1.00 all = 3.5658 } Zip/UnZip{ index = 0.0957 value = 0.2902 dict = 1.00 all = 0.2804 } ---------------------------- total value len = 1.249402 GB avg = 0.121 KB (by entry num) total key len = 0.289700 GB avg = 0.028 KB total ukey len = 0.206928 GB avg = 0.020 KB total ukey num = 0.010346417 Billion total entry num = 0.010346417 Billion write speed all = 0.026971985 MB/sec (with seq num) write speed all = 0.025521459 MB/sec (without seq num) 2024/11/23-00:43:26.106929 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236949: 926124 keys, 37639866 bytes, temperature: kWarm 2024/11/23-00:43:26.106995 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293806106963, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236949, "file_size": 37639866, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3682496154, "table_properties": {"data_size": 31423024, "index_size": 1773120, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25931472, "raw_average_key_size": 28, "raw_value_size": 108286312, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926124, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293802, "oldest_key_time": 0, "file_creation_time": 1732293802, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236949, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:43:26.144621 140024751826496 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236950.sst) /dev/shm/ToplingTemp/Topling-354810-bxCPbi 2024/11/23-00:43:26.567677 140024751826496 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236950.sst): first pass time = 0.42's, 243.051'MB/sec 2024/11/23-00:43:27.617615 140024751826496 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236950.sst): old prealloc_size = 147639500, real_size = 25935776 2024/11/23-00:43:27.618067 140024751826496 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236950.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 711037 prefix = 4 raw-key = 0.0114 GB zip-key = 0.0014 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.0829 GB zip-val = 0.0244 GB avg-val = 116.59 avg-zval = 34.32 2024/11/23-00:43:27.618654 140024751826496 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236950.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0018 GB, waited 0.000 sec, Key+Value = 0.103 GB 2024/11/23-00:43:28.336464 140024751826496 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00/236950.sst): second pass time = 1.05's, 79.048'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.72's, 40.336'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3186.023'MB/sec (index lex order gen) rebuild zvType time = 0.62's, 0.287'MB/sec write SST data time = 0.10's, 294.839'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.833'MB, throughput = 25895.238'MB/sec zip my value time = 1.05's, unzip length = 0.083'GB zip my value throughput = 79.048'MB/sec zip pipeline throughput = 79.048'MB/sec entries = 711037 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.59 avg-zval = 34.32 usrkeys = 711037 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.59 avg-zval = 34.32 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 711037 bytes = 2399760 } seq expand size = 4977259 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0142 GB value = 0.0829 GB dict = 0.83 MB all = 0.1028 GB } __ZipSize{ index = 0.0014 GB value = 0.0244 GB dict = 0.83 MB all = 0.0290 GB } UnZip/Zip{ index = 10.5098 value = 3.3970 dict = 1.00 all = 3.5459 } Zip/UnZip{ index = 0.0951 value = 0.2944 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 1.337283 GB avg = 0.121 KB (by entry num) total key len = 0.309609 GB avg = 0.028 KB total ukey len = 0.221149 GB avg = 0.020 KB total ukey num = 0.011057454 Billion total entry num = 0.011057454 Billion write speed all = 0.028859799 MB/sec (with seq num) write speed all = 0.027309650 MB/sec (without seq num) 2024/11/23-00:43:28.362543 140024751826496 [compaction_job.cc:1948] [default] [JOB 15523] Generated table #236950: 711037 keys, 28995018 bytes, temperature: kWarm 2024/11/23-00:43:28.362621 140024751826496 EVENT_LOG_v1 {"time_micros": 1732293808362573, "cf_name": "default", "job": 15523, "event": "table_file_creation", "file_number": 236950, "file_size": 28995018, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3610601794, "largest_seqno": 3682496167, "table_properties": {"data_size": 24404752, "index_size": 1353088, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 19909036, "raw_average_key_size": 28, "raw_value_size": 82903310, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 711037, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732293806, "oldest_key_time": 0, "file_creation_time": 1732293806, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 236950, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-00:43:28.384450 140024751826496 [compaction_job.cc:703] job-15523: subcompact[0], size: 0.436138 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-15523/att-00 [236939,236940,236941,236942,236943,236944,236945,236946,236947,236948,236949,236950] 2024/11/23-00:43:28.462140 140024751826496 (Original Log Time 2024/11/23-00:42:50.920293) EVENT_LOG_v1 {"time_micros": 1732293770920253, "job": 15523, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [236878], "files_L3": [236935, 236936, 236832, 236833, 236834, 236835, 236836, 236837, 236838, 236839, 236840, 236793, 236913], "score": 0.99973, "input_data_size": 451589234, "oldest_snapshot_seqno": 3682991372} 2024/11/23-00:43:28.462144 140024751826496 (Original Log Time 2024/11/23-00:43:28.461434) [compaction_job.cc:2022] [default] [JOB 15523] Compacted 1@2 + 13@3 files to L3 => 436137720 bytes 2024/11/23-00:43:28.462145 140024751826496 (Original Log Time 2024/11/23-00:43:28.462073) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 12 0 0 0] max score 0.02, MB/sec: 12.1 rd, 11.6 wr, level 3, files in(1, 13) out(12 +0 blob) MB in(21.3, 409.4 +0.0 blob) out(415.9 +0.0 blob), read-write-amplify(39.7) write-amplify(19.5) OK, records in: 11131041, records dropped: 73587 output_compression: Snappy 2024/11/23-00:43:28.462152 140024751826496 (Original Log Time 2024/11/23-00:43:28.462120) EVENT_LOG_v1 {"time_micros": 1732293808462098, "cf": "default", "job": 15523, "event": "compaction_finished", "compaction_time_micros": 37464167, "compaction_time_cpu_micros": 16966215, "output_level": 3, "num_output_files": 12, "total_output_size": 436137720, "num_input_records": 11131041, "num_output_records": 11057454, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 271231255, "file_range_sync_nanos": 0, "file_fsync_nanos": 11440, "file_prepare_write_nanos": 26535, "lsm_state": [0, 0, 0, 12, 0, 0, 0]} 2024/11/23-00:43:28.593030 140024751826496 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732293770,"executesMs":37654,"compactionJobId":15523,"attempt":0,"compactionInputRawBytes":1589657347,"compactionInputZipBytes":451589234,"compactionOutputRawBytes":0,"compactionOutputZipBytes":436137720,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-d734861159d8b8f12f4af7d274890e74-087835ba82ae2515-00"}}, response = 2024/11/23-00:43:28.593108 140024751826496 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-15523/att-00: olev 3, work 37.655 s, result 6.308 ms, install 0.735 ms, input{raw 1.480 GiB zip 430.669 MiB}