2024/11/22-19:14:35.123103 140024718255680 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00: fork to child time = 0.021281 sec 2024/11/22-19:14:35.124310 140024718255680 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00 2024/11/22-19:14:35.133609 140024718255680 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00 2024/11/22-19:14:35.135740 140024718255680 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/22-19:14:35.136165 140024718255680 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/22-19:14:35.182069 140024718255680 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08558/att-00/MANIFEST-206347 2024/11/22-19:14:35.218569 140024718255680 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08558/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 222648, last_sequence is 3494645927, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 222491 2024/11/22-19:14:35.218594 140024718255680 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/22-19:14:35.218724 140024718255680 [version_set.cc:5721] Creating manifest 222648 2024/11/22-19:14:35.249355 140024718255680 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/22-19:14:35.268689 140024718255680 [compaction_job.cc:2437] [default] [JOB 8558] Compacting 2@2 + 11@3 files to L3, score 0.97, subcompactions 1 : 1 2024/11/22-19:14:35.268715 140024718255680 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 6494 Base level 2, inputs: [222635(27MB) 222640(3486KB)], [222592(34MB) 222593(34MB) 222595(35MB) 222596(35MB) 222597(36MB) 222598(36MB) 222599(36MB) 222600(34MB) 222552(35MB) 222553(35MB) 222554(36MB)] 2024/11/22-19:14:35.277511 140024718255680 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:14:35.277556 140024718255680 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:14:35.277584 140024718255680 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/22-19:14:35.277593 140024718255680 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-19:14:35.277608 140024718255680 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/22-19:14:35.767654 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222649.sst) /dev/shm/ToplingTemp/Topling-255092-BZ15H3 2024/11/22-19:14:36.627271 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222649.sst): first pass time = 0.86's, 156.145'MB/sec 2024/11/22-19:14:40.181549 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222649.sst): old prealloc_size = 147639500, real_size = 32483008 2024/11/22-19:14:40.182150 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222649.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925213 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.92 raw-val = 0.1083 GB zip-val = 0.0305 GB avg-val = 117.07 avg-zval = 32.94 2024/11/22-19:14:40.182973 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222649.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:14:41.333564 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222649.sst): second pass time = 3.55's, 30.480'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.15's, 31.750'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3008.610'MB/sec (index lex order gen) rebuild zvType time = 0.92's, 0.252'MB/sec write SST data time = 0.23's, 157.587'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28294.737'MB/sec zip my value time = 3.55's, unzip length = 0.108'GB zip my value throughput = 30.480'MB/sec zip pipeline throughput = 123.904'MB/sec entries = 925213 avg-key = 28.00 avg-zkey = 1.92 avg-val = 117.07 avg-zval = 32.94 usrkeys = 925213 avg-key = 20.00 avg-zkey = 1.92 avg-val = 124.07 avg-zval = 32.94 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925213 bytes = 3238256 } seq expand size = 6476491 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.4424 value = 3.5536 dict = 1.00 all = 3.6695 } Zip/UnZip{ index = 0.0958 value = 0.2814 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 0.114788 GB avg = 0.124 KB (by entry num) total key len = 0.025906 GB avg = 0.028 KB total ukey len = 0.018504 GB avg = 0.020 KB total ukey num = 0.000925213 Billion total entry num = 0.000925213 Billion write speed all = 0.003768100 MB/sec (with seq num) write speed all = 0.003569867 MB/sec (without seq num) 2024/11/22-19:14:41.372460 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222649: 925213 keys, 36576336 bytes, temperature: kWarm 2024/11/22-19:14:41.372568 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274081372499, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222649, "file_size": 36576336, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701122, "largest_seqno": 3494592166, "table_properties": {"data_size": 30479472, "index_size": 1772032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25905964, "raw_average_key_size": 28, "raw_value_size": 108311774, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925213, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274075, "oldest_key_time": 0, "file_creation_time": 1732274075, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222649, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:14:41.437233 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222650.sst) /dev/shm/ToplingTemp/Topling-255092-S4Bvj5 2024/11/22-19:14:42.781811 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222650.sst): first pass time = 1.34's, 99.825'MB/sec 2024/11/22-19:14:45.324738 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222650.sst): old prealloc_size = 147639500, real_size = 32557872 2024/11/22-19:14:45.325302 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222650.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925141 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0306 GB avg-val = 117.08 avg-zval = 33.04 2024/11/22-19:14:45.326221 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222650.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:14:46.427106 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222650.sst): second pass time = 2.54's, 42.614'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 33.248'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3244.216'MB/sec (index lex order gen) rebuild zvType time = 0.97's, 0.238'MB/sec write SST data time = 0.13's, 280.399'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 20605.377'MB/sec zip my value time = 2.54's, unzip length = 0.108'GB zip my value throughput = 42.614'MB/sec zip pipeline throughput = 86.484'MB/sec entries = 925141 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.08 avg-zval = 33.04 usrkeys = 925141 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.08 avg-zval = 33.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 925141 bytes = 3238016 } seq expand size = 6475987 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0306 GB dict = 1.08 MB all = 0.0367 GB } UnZip/Zip{ index = 10.5130 value = 3.5436 dict = 1.00 all = 3.6620 } Zip/UnZip{ index = 0.0951 value = 0.2822 dict = 1.00 all = 0.2731 } ---------------------------- total value len = 0.229578 GB avg = 0.124 KB (by entry num) total key len = 0.051810 GB avg = 0.028 KB total ukey len = 0.037007 GB avg = 0.020 KB total ukey num = 0.001850354 Billion total entry num = 0.001850354 Billion write speed all = 0.007535159 MB/sec (with seq num) write speed all = 0.007138761 MB/sec (without seq num) 2024/11/22-19:14:46.458348 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222650: 925141 keys, 36651264 bytes, temperature: kWarm 2024/11/22-19:14:46.458429 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274086458397, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222650, "file_size": 36651264, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3326330223, "largest_seqno": 3494592437, "table_properties": {"data_size": 30566432, "index_size": 1760000, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25903948, "raw_average_key_size": 28, "raw_value_size": 108313782, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925141, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274081, "oldest_key_time": 0, "file_creation_time": 1732274081, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222650, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:14:46.496773 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222651.sst) /dev/shm/ToplingTemp/Topling-255092-XhZvG5 2024/11/22-19:14:47.916091 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222651.sst): first pass time = 1.42's, 94.574'MB/sec 2024/11/22-19:14:50.657408 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222651.sst): old prealloc_size = 147639500, real_size = 33320384 2024/11/22-19:14:50.658001 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222651.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923632 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0313 GB avg-val = 117.32 avg-zval = 33.92 2024/11/22-19:14:50.658729 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222651.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:14:51.755480 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222651.sst): second pass time = 2.74's, 39.536'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.10's, 34.070'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3069.840'MB/sec (index lex order gen) rebuild zvType time = 0.87's, 0.266'MB/sec write SST data time = 0.23's, 162.926'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 26064.087'MB/sec zip my value time = 2.74's, unzip length = 0.108'GB zip my value throughput = 39.536'MB/sec zip pipeline throughput = 133.358'MB/sec entries = 923632 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.32 avg-zval = 33.92 usrkeys = 923632 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.32 avg-zval = 33.92 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923632 bytes = 3232720 } seq expand size = 6465424 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0313 GB dict = 1.08 MB all = 0.0374 GB } UnZip/Zip{ index = 10.5142 value = 3.4583 dict = 1.00 all = 3.5877 } Zip/UnZip{ index = 0.0951 value = 0.2892 dict = 1.00 all = 0.2787 } ---------------------------- total value len = 0.344400 GB avg = 0.124 KB (by entry num) total key len = 0.077672 GB avg = 0.028 KB total ukey len = 0.055480 GB avg = 0.020 KB total ukey num = 0.002773986 Billion total entry num = 0.002773986 Billion write speed all = 0.011300840 MB/sec (with seq num) write speed all = 0.010706658 MB/sec (without seq num) 2024/11/22-19:14:51.793609 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222651: 923632 keys, 37410640 bytes, temperature: kWarm 2024/11/22-19:14:51.793670 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274091793640, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222651, "file_size": 37410640, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701165, "largest_seqno": 3494593838, "table_properties": {"data_size": 31332336, "index_size": 1756928, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25861696, "raw_average_key_size": 28, "raw_value_size": 108356160, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923632, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274086, "oldest_key_time": 0, "file_creation_time": 1732274086, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222651, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:14:51.831373 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222652.sst) /dev/shm/ToplingTemp/Topling-255092-IKZoB4 2024/11/22-19:14:53.309835 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222652.sst): first pass time = 1.48's, 90.784'MB/sec 2024/11/22-19:14:55.018318 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222652.sst): old prealloc_size = 147639500, real_size = 33726816 2024/11/22-19:14:55.019065 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222652.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923188 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.39 avg-zval = 34.38 2024/11/22-19:14:55.019816 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222652.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:14:56.080757 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222652.sst): second pass time = 1.71's, 63.469'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.06's, 35.486'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2412.446'MB/sec (index lex order gen) rebuild zvType time = 0.83's, 0.278'MB/sec write SST data time = 0.23's, 163.277'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 38808.386'MB/sec zip my value time = 1.71's, unzip length = 0.108'GB zip my value throughput = 63.469'MB/sec zip pipeline throughput = 65.058'MB/sec entries = 923188 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 34.38 usrkeys = 923188 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 34.38 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923188 bytes = 3115776 } seq expand size = 6462316 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0377 GB } UnZip/Zip{ index = 10.5118 value = 3.4143 dict = 1.00 all = 3.5601 } Zip/UnZip{ index = 0.0951 value = 0.2929 dict = 1.00 all = 0.2809 } ---------------------------- total value len = 0.459230 GB avg = 0.124 KB (by entry num) total key len = 0.103521 GB avg = 0.028 KB total ukey len = 0.073943 GB avg = 0.020 KB total ukey num = 0.003697174 Billion total entry num = 0.003697174 Billion write speed all = 0.015065766 MB/sec (with seq num) write speed all = 0.014273931 MB/sec (without seq num) 2024/11/22-19:14:56.119776 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222652: 923188 keys, 37700564 bytes, temperature: kWarm 2024/11/22-19:14:56.119879 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274096119821, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222652, "file_size": 37700564, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3369353416, "largest_seqno": 3494593245, "table_properties": {"data_size": 31739344, "index_size": 1756480, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25849264, "raw_average_key_size": 28, "raw_value_size": 108368508, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923188, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274091, "oldest_key_time": 0, "file_creation_time": 1732274091, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222652, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:14:56.196866 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222653.sst) /dev/shm/ToplingTemp/Topling-255092-zB2d34 2024/11/22-19:14:57.604965 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222653.sst): first pass time = 1.41's, 95.322'MB/sec 2024/11/22-19:14:59.764075 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222653.sst): old prealloc_size = 147639500, real_size = 35541632 2024/11/22-19:14:59.764728 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222653.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 927797 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1082 GB zip-val = 0.0335 GB avg-val = 116.66 avg-zval = 36.15 2024/11/22-19:14:59.765212 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222653.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:15:00.830992 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222653.sst): second pass time = 2.16's, 50.149'MB/sec, value only(80.6% of KV) wait indexing time = 0.00's, remap KeyValue time = 1.07's, 35.474'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2798.964'MB/sec (index lex order gen) rebuild zvType time = 0.90's, 0.257'MB/sec write SST data time = 0.16's, 231.699'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 29390.405'MB/sec zip my value time = 2.16's, unzip length = 0.108'GB zip my value throughput = 50.149'MB/sec zip pipeline throughput = 50.149'MB/sec entries = 927797 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.66 avg-zval = 36.15 usrkeys = 927797 avg-key = 20.00 avg-zkey = 1.90 avg-val = 119.69 avg-zval = 36.15 TagRS{ kinds = RS_Key0_TagN bytes = 145080 } TagArray{ size = 401111 bytes = 1303632 } seq expand size = 2807777 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0186 GB value = 0.1082 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0335 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5088 value = 3.2268 dict = 1.00 all = 3.5462 } Zip/UnZip{ index = 0.0952 value = 0.3099 dict = 1.00 all = 0.2820 } ---------------------------- total value len = 0.570278 GB avg = 0.123 KB (by entry num) total key len = 0.129499 GB avg = 0.028 KB total ukey len = 0.092499 GB avg = 0.020 KB total ukey num = 0.004624971 Billion total entry num = 0.004624971 Billion write speed all = 0.018731782 MB/sec (with seq num) write speed all = 0.017741364 MB/sec (without seq num) 2024/11/22-19:15:00.867470 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222653: 927797 keys, 37848019 bytes, temperature: kWarm 2024/11/22-19:15:00.867539 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274100867504, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222653, "file_size": 37848019, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3494592814, "table_properties": {"data_size": 33543728, "index_size": 1765760, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25978316, "raw_average_key_size": 28, "raw_value_size": 108239530, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 927797, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274096, "oldest_key_time": 0, "file_creation_time": 1732274096, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222653, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:15:00.906829 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222654.sst) /dev/shm/ToplingTemp/Topling-255092-UOAmf4 2024/11/22-19:15:01.914793 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222654.sst): first pass time = 1.01's, 133.163'MB/sec 2024/11/22-19:15:03.366399 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222654.sst): old prealloc_size = 147639500, real_size = 34379520 2024/11/22-19:15:03.366961 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222654.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 925017 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0017 GB avg-key = 16.00 avg-zkey = 1.89 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 117.10 avg-zval = 35.03 2024/11/22-19:15:03.367595 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222654.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:15:04.269166 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222654.sst): second pass time = 1.45's, 74.653'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.90's, 42.131'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3239.400'MB/sec (index lex order gen) rebuild zvType time = 0.77's, 0.302'MB/sec write SST data time = 0.14's, 279.898'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 33252.461'MB/sec zip my value time = 1.45's, unzip length = 0.108'GB zip my value throughput = 74.653'MB/sec zip pipeline throughput = 74.653'MB/sec entries = 925017 avg-key = 28.00 avg-zkey = 1.89 avg-val = 117.10 avg-zval = 35.03 usrkeys = 925017 avg-key = 20.00 avg-zkey = 1.89 avg-val = 122.83 avg-zval = 35.03 TagRS{ kinds = RS_Key0_TagN bytes = 144640 } TagArray{ size = 757404 bytes = 2650928 } seq expand size = 5301828 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0017 GB value = 0.0324 GB dict = 1.08 MB all = 0.0380 GB } UnZip/Zip{ index = 10.5851 value = 3.3431 dict = 1.00 all = 3.5289 } Zip/UnZip{ index = 0.0945 value = 0.2991 dict = 1.00 all = 0.2834 } ---------------------------- total value len = 0.683897 GB avg = 0.123 KB (by entry num) total key len = 0.155400 GB avg = 0.028 KB total ukey len = 0.111000 GB avg = 0.020 KB total ukey num = 0.005549988 Billion total entry num = 0.005549988 Billion write speed all = 0.022464406 MB/sec (with seq num) write speed all = 0.021276009 MB/sec (without seq num) 2024/11/22-19:15:04.309893 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222654: 925017 keys, 38034198 bytes, temperature: kWarm 2024/11/22-19:15:04.309954 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274104309924, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222654, "file_size": 38034198, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3494593922, "table_properties": {"data_size": 32400304, "index_size": 1747776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25900476, "raw_average_key_size": 28, "raw_value_size": 108317322, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 925017, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274100, "oldest_key_time": 0, "file_creation_time": 1732274100, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222654, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:15:04.345909 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222655.sst) /dev/shm/ToplingTemp/Topling-255092-nwDef2 2024/11/22-19:15:05.224685 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222655.sst): first pass time = 0.88's, 152.739'MB/sec 2024/11/22-19:15:06.666440 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222655.sst): old prealloc_size = 147639500, real_size = 33681024 2024/11/22-19:15:06.667034 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222655.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923223 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0317 GB avg-val = 117.38 avg-zval = 34.33 2024/11/22-19:15:06.667828 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222655.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:15:07.622827 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222655.sst): second pass time = 1.44's, 75.203'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.96's, 39.494'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3053.920'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.284'MB/sec write SST data time = 0.14's, 262.182'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 35269.476'MB/sec zip my value time = 1.44's, unzip length = 0.108'GB zip my value throughput = 75.203'MB/sec zip pipeline throughput = 75.203'MB/sec entries = 923223 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.38 avg-zval = 34.33 usrkeys = 923223 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.38 avg-zval = 34.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923223 bytes = 3231296 } seq expand size = 6462561 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0317 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.5130 value = 3.4192 dict = 1.00 all = 3.5535 } Zip/UnZip{ index = 0.0951 value = 0.2925 dict = 1.00 all = 0.2814 } ---------------------------- total value len = 0.798727 GB avg = 0.123 KB (by entry num) total key len = 0.181250 GB avg = 0.028 KB total ukey len = 0.129464 GB avg = 0.020 KB total ukey num = 0.006473211 Billion total entry num = 0.006473211 Billion write speed all = 0.026227470 MB/sec (with seq num) write speed all = 0.024841511 MB/sec (without seq num) 2024/11/22-19:15:07.660948 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222655: 923223 keys, 37770290 bytes, temperature: kWarm 2024/11/22-19:15:07.661015 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274107660980, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222655, "file_size": 37770290, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3273701550, "largest_seqno": 3494593010, "table_properties": {"data_size": 31693680, "index_size": 1756352, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25850244, "raw_average_key_size": 28, "raw_value_size": 108367618, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923223, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274104, "oldest_key_time": 0, "file_creation_time": 1732274104, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222655, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:15:07.696326 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222656.sst) /dev/shm/ToplingTemp/Topling-255092-Sx8lF5 2024/11/22-19:15:08.614247 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222656.sst): first pass time = 0.92's, 146.226'MB/sec 2024/11/22-19:15:10.094500 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222656.sst): old prealloc_size = 147639500, real_size = 32488640 2024/11/22-19:15:10.095052 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222656.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923142 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0305 GB avg-val = 117.39 avg-zval = 33.04 2024/11/22-19:15:10.095844 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222656.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:15:10.976310 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222656.sst): second pass time = 1.48's, 73.246'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.88's, 41.481'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3306.029'MB/sec (index lex order gen) rebuild zvType time = 0.75's, 0.307'MB/sec write SST data time = 0.13's, 283.514'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 33895.271'MB/sec zip my value time = 1.48's, unzip length = 0.108'GB zip my value throughput = 73.246'MB/sec zip pipeline throughput = 73.246'MB/sec entries = 923142 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.39 avg-zval = 33.04 usrkeys = 923142 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.39 avg-zval = 33.04 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923142 bytes = 3231008 } seq expand size = 6461994 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0305 GB dict = 1.08 MB all = 0.0366 GB } UnZip/Zip{ index = 10.5155 value = 3.5529 dict = 1.00 all = 3.6694 } Zip/UnZip{ index = 0.0951 value = 0.2815 dict = 1.00 all = 0.2725 } ---------------------------- total value len = 0.913559 GB avg = 0.124 KB (by entry num) total key len = 0.207098 GB avg = 0.028 KB total ukey len = 0.147927 GB avg = 0.020 KB total ukey num = 0.007396353 Billion total entry num = 0.007396353 Billion write speed all = 0.029989841 MB/sec (with seq num) write speed all = 0.028406374 MB/sec (without seq num) 2024/11/22-19:15:11.011893 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222656: 923142 keys, 36577592 bytes, temperature: kWarm 2024/11/22-19:15:11.011959 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274111011927, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222656, "file_size": 36577592, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3335391241, "largest_seqno": 3494592036, "table_properties": {"data_size": 30501872, "index_size": 1755776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847976, "raw_average_key_size": 28, "raw_value_size": 108369832, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923142, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274107, "oldest_key_time": 0, "file_creation_time": 1732274107, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222656, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:15:11.052738 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222657.sst) /dev/shm/ToplingTemp/Topling-255092-YOjO21 2024/11/22-19:15:11.960400 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222657.sst): first pass time = 0.91's, 147.879'MB/sec 2024/11/22-19:15:13.391729 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222657.sst): old prealloc_size = 147639500, real_size = 33605072 2024/11/22-19:15:13.392258 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222657.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 923121 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1084 GB zip-val = 0.0316 GB avg-val = 117.40 avg-zval = 34.25 2024/11/22-19:15:13.392993 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222657.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:15:14.327070 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222657.sst): second pass time = 1.43's, 75.758'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.94's, 40.177'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3455.073'MB/sec (index lex order gen) rebuild zvType time = 0.81's, 0.286'MB/sec write SST data time = 0.13's, 293.789'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.083'MB, throughput = 34312.031'MB/sec zip my value time = 1.43's, unzip length = 0.108'GB zip my value throughput = 75.758'MB/sec zip pipeline throughput = 75.758'MB/sec entries = 923121 avg-key = 28.00 avg-zkey = 1.90 avg-val = 117.40 avg-zval = 34.25 usrkeys = 923121 avg-key = 20.00 avg-zkey = 1.90 avg-val = 124.40 avg-zval = 34.25 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 923121 bytes = 3115552 } seq expand size = 6461847 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1084 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0316 GB dict = 1.08 MB all = 0.0376 GB } UnZip/Zip{ index = 10.5126 value = 3.4275 dict = 1.00 all = 3.5717 } Zip/UnZip{ index = 0.0951 value = 0.2918 dict = 1.00 all = 0.2800 } ---------------------------- total value len = 1.028391 GB avg = 0.124 KB (by entry num) total key len = 0.232945 GB avg = 0.028 KB total ukey len = 0.166389 GB avg = 0.020 KB total ukey num = 0.008319474 Billion total entry num = 0.008319474 Billion write speed all = 0.033751536 MB/sec (with seq num) write speed all = 0.031970599 MB/sec (without seq num) 2024/11/22-19:15:14.365475 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222657: 923121 keys, 37578554 bytes, temperature: kWarm 2024/11/22-19:15:14.365537 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274114365507, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222657, "file_size": 37578554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268533, "largest_seqno": 3494593791, "table_properties": {"data_size": 31617856, "index_size": 1756224, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25847388, "raw_average_key_size": 28, "raw_value_size": 108370426, "raw_average_value_size": 117, "num_data_blocks": 1, "num_entries": 923121, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274111, "oldest_key_time": 0, "file_creation_time": 1732274111, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222657, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:15:14.403827 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222658.sst) /dev/shm/ToplingTemp/Topling-255092-kIwaa5 2024/11/22-19:15:15.208581 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222658.sst): first pass time = 0.80's, 166.790'MB/sec 2024/11/22-19:15:16.714881 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222658.sst): old prealloc_size = 147639500, real_size = 33859728 2024/11/22-19:15:16.715406 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222658.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926908 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.91 raw-val = 0.1083 GB zip-val = 0.0319 GB avg-val = 116.80 avg-zval = 34.36 2024/11/22-19:15:16.716139 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222658.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:15:17.621222 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222658.sst): second pass time = 1.51's, 71.906'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.91's, 41.753'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3508.270'MB/sec (index lex order gen) rebuild zvType time = 0.78's, 0.297'MB/sec write SST data time = 0.13's, 298.899'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.080'MB, throughput = 35219.359'MB/sec zip my value time = 1.51's, unzip length = 0.108'GB zip my value throughput = 71.906'MB/sec zip pipeline throughput = 71.906'MB/sec entries = 926908 avg-key = 28.00 avg-zkey = 1.91 avg-val = 116.80 avg-zval = 34.36 usrkeys = 926908 avg-key = 20.00 avg-zkey = 1.91 avg-val = 123.80 avg-zval = 34.36 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926908 bytes = 3128336 } seq expand size = 6488356 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0319 GB dict = 1.08 MB all = 0.0378 GB } UnZip/Zip{ index = 10.4449 value = 3.3989 dict = 1.00 all = 3.5468 } Zip/UnZip{ index = 0.0957 value = 0.2942 dict = 1.00 all = 0.2819 } ---------------------------- total value len = 1.143144 GB avg = 0.124 KB (by entry num) total key len = 0.258899 GB avg = 0.028 KB total ukey len = 0.184928 GB avg = 0.020 KB total ukey num = 0.009246382 Billion total entry num = 0.009246382 Billion write speed all = 0.037513324 MB/sec (with seq num) write speed all = 0.035534140 MB/sec (without seq num) 2024/11/22-19:15:17.655170 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222658: 926908 keys, 37842088 bytes, temperature: kWarm 2024/11/22-19:15:17.655234 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274117655202, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222658, "file_size": 37842088, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268543, "largest_seqno": 3494592872, "table_properties": {"data_size": 31852992, "index_size": 1774848, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25953424, "raw_average_key_size": 28, "raw_value_size": 108264436, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926908, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274114, "oldest_key_time": 0, "file_creation_time": 1732274114, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222658, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:15:17.691163 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222659.sst) /dev/shm/ToplingTemp/Topling-255092-sryoD4 2024/11/22-19:15:18.797956 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222659.sst): first pass time = 1.11's, 121.272'MB/sec 2024/11/22-19:15:20.160802 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222659.sst): old prealloc_size = 147639500, real_size = 34436464 2024/11/22-19:15:20.161326 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222659.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 926441 prefix = 4 raw-key = 0.0148 GB zip-key = 0.0018 GB avg-key = 16.00 avg-zkey = 1.90 raw-val = 0.1083 GB zip-val = 0.0324 GB avg-val = 116.87 avg-zval = 35.02 2024/11/22-19:15:20.162053 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222659.sst: sumWaitingMem = 0.002 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0023 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/22-19:15:21.080543 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222659.sst): second pass time = 1.36's, 79.537'MB/sec, value only(80.7% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.92's, 41.770'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 3494.579'MB/sec (index lex order gen) rebuild zvType time = 0.79's, 0.291'MB/sec write SST data time = 0.12's, 309.114'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.081'MB, throughput = 28605.919'MB/sec zip my value time = 1.36's, unzip length = 0.108'GB zip my value throughput = 79.537'MB/sec zip pipeline throughput = 79.537'MB/sec entries = 926441 avg-key = 28.00 avg-zkey = 1.90 avg-val = 116.87 avg-zval = 35.02 usrkeys = 926441 avg-key = 20.00 avg-zkey = 1.90 avg-val = 123.87 avg-zval = 35.02 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 926441 bytes = 3126752 } seq expand size = 6485087 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0185 GB value = 0.1083 GB dict = 1.08 MB all = 0.1342 GB } __ZipSize{ index = 0.0018 GB value = 0.0324 GB dict = 1.08 MB all = 0.0384 GB } UnZip/Zip{ index = 10.5098 value = 3.3376 dict = 1.00 all = 3.4936 } Zip/UnZip{ index = 0.0951 value = 0.2996 dict = 1.00 all = 0.2862 } ---------------------------- total value len = 1.257907 GB avg = 0.124 KB (by entry num) total key len = 0.284839 GB avg = 0.028 KB total ukey len = 0.203456 GB avg = 0.020 KB total ukey num = 0.010172823 Billion total entry num = 0.010172823 Billion write speed all = 0.041274179 MB/sec (with seq num) write speed all = 0.039096892 MB/sec (without seq num) 2024/11/22-19:15:21.119933 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222659: 926441 keys, 38417984 bytes, temperature: kWarm 2024/11/22-19:15:21.119991 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274121119961, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222659, "file_size": 38417984, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3405268578, "largest_seqno": 3494594064, "table_properties": {"data_size": 32441632, "index_size": 1763008, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 25940348, "raw_average_key_size": 28, "raw_value_size": 108277506, "raw_average_value_size": 116, "num_data_blocks": 1, "num_entries": 926441, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274117, "oldest_key_time": 0, "file_creation_time": 1732274117, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222659, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:15:21.159443 140024718255680 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222660.sst) /dev/shm/ToplingTemp/Topling-255092-IUjZ95 2024/11/22-19:15:21.240009 140024718255680 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222660.sst): first pass time = 0.08's, 235.419'MB/sec 2024/11/22-19:15:21.406715 140024718255680 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222660.sst): old prealloc_size = 147639500, real_size = 5208832 2024/11/22-19:15:21.406855 140024718255680 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222660.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 133278 prefix = 4 raw-key = 0.0021 GB zip-key = 0.0003 GB avg-key = 16.00 avg-zkey = 1.93 raw-val = 0.0152 GB zip-val = 0.0049 GB avg-val = 114.23 avg-zval = 36.90 2024/11/22-19:15:21.406977 140024718255680 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222660.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.019 GB 2024/11/22-19:15:21.526331 140024718255680 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00/222660.sst): second pass time = 0.17's, 91.601'MB/sec, value only(80.3% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.12's, 48.078'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2198.415'MB/sec (index lex order gen) rebuild zvType time = 0.10's, 0.331'MB/sec write SST data time = 0.02's, 304.796'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.159'MB, throughput = 3933.203'MB/sec zip my value time = 0.17's, unzip length = 0.015'GB zip my value throughput = 91.601'MB/sec zip pipeline throughput = 91.601'MB/sec entries = 133278 avg-key = 28.00 avg-zkey = 1.93 avg-val = 114.23 avg-zval = 36.90 usrkeys = 133278 avg-key = 20.00 avg-zkey = 1.93 avg-val = 120.82 avg-zval = 36.90 TagRS{ kinds = RS_Key0_TagN bytes = 20920 } TagArray{ size = 125469 bytes = 392112 } seq expand size = 878283 multi value expand size = 0 cnt WriteAppend = 15 UnZipSize{ index = 0.0027 GB value = 0.0152 GB dict = 0.16 MB all = 0.0190 GB } __ZipSize{ index = 0.0003 GB value = 0.0049 GB dict = 0.16 MB all = 0.0058 GB } UnZip/Zip{ index = 10.3786 value = 3.0954 dict = 1.00 all = 3.2967 } Zip/UnZip{ index = 0.0964 value = 0.3231 dict = 1.00 all = 0.3033 } ---------------------------- total value len = 1.274010 GB avg = 0.124 KB (by entry num) total key len = 0.288571 GB avg = 0.028 KB total ukey len = 0.206122 GB avg = 0.020 KB total ukey num = 0.010306101 Billion total entry num = 0.010306101 Billion write speed all = 0.041804340 MB/sec (with seq num) write speed all = 0.039598554 MB/sec (without seq num) 2024/11/22-19:15:21.537838 140024718255680 [compaction_job.cc:1948] [default] [JOB 8558] Generated table #222660: 133278 keys, 5750265 bytes, temperature: kWarm 2024/11/22-19:15:21.537893 140024718255680 EVENT_LOG_v1 {"time_micros": 1732274121537865, "cf_name": "default", "job": 8558, "event": "table_file_creation", "file_number": 222660, "file_size": 5750265, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 3494594114, "table_properties": {"data_size": 4918512, "index_size": 256832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 3731784, "raw_average_key_size": 28, "raw_value_size": 15224928, "raw_average_value_size": 114, "num_data_blocks": 1, "num_entries": 133278, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 28, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732274121, "oldest_key_time": 0, "file_creation_time": 1732274121, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 222660, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/22-19:15:21.549256 140024718255680 [compaction_job.cc:703] job-08558: subcompact[0], size: 0.418158 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-08558/att-00 [222649,222650,222651,222652,222653,222654,222655,222656,222657,222658,222659,222660] 2024/11/22-19:15:21.629047 140024718255680 (Original Log Time 2024/11/22-19:14:35.268807) EVENT_LOG_v1 {"time_micros": 1732274075268735, "job": 8558, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [222635, 222640], "files_L3": [222592, 222593, 222595, 222596, 222597, 222598, 222599, 222600, 222552, 222553, 222554], "score": 0.971498, "input_data_size": 443929414, "oldest_snapshot_seqno": 3494645927} 2024/11/22-19:15:21.629050 140024718255680 (Original Log Time 2024/11/22-19:15:21.628373) [compaction_job.cc:2022] [default] [JOB 8558] Compacted 2@2 + 11@3 files to L3 => 418157794 bytes 2024/11/22-19:15:21.629052 140024718255680 (Original Log Time 2024/11/22-19:15:21.628985) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 12 0 0 0] max score 0.02, MB/sec: 9.6 rd, 9.0 wr, level 3, files in(2, 11) out(12 +0 blob) MB in(30.5, 392.9 +0.0 blob) out(398.8 +0.0 blob), read-write-amplify(27.0) write-amplify(13.1) OK, records in: 10396164, records dropped: 90063 output_compression: Snappy 2024/11/22-19:15:21.629058 140024718255680 (Original Log Time 2024/11/22-19:15:21.629029) EVENT_LOG_v1 {"time_micros": 1732274121629008, "cf": "default", "job": 8558, "event": "compaction_finished", "compaction_time_micros": 46280454, "compaction_time_cpu_micros": 19337503, "output_level": 3, "num_output_files": 12, "total_output_size": 418157794, "num_input_records": 10396164, "num_output_records": 10306101, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 279010437, "file_range_sync_nanos": 0, "file_fsync_nanos": 11098, "file_prepare_write_nanos": 27636, "lsm_state": [0, 0, 0, 12, 0, 0, 0]} 2024/11/22-19:15:21.753259 140024718255680 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732274075,"executesMs":46500,"compactionJobId":8558,"attempt":0,"compactionInputRawBytes":1508141596,"compactionInputZipBytes":443929414,"compactionOutputRawBytes":0,"compactionOutputZipBytes":418157794,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-5d6ff8db4030487a728d2f2418485047-5e6833ba9de9032d-00"}}, response = 2024/11/22-19:15:21.753325 140024718255680 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-08558/att-00: olev 3, work 46.500 s, result 4.842 ms, install 0.702 ms, input{raw 1.405 GiB zip 423.364 MiB}