2024/11/23-14:59:09.790277 139957668369984 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00: fork to child time = 0.008382 sec 2024/11/23-14:59:09.791179 139957668369984 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00 2024/11/23-14:59:09.799197 139957668369984 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00 2024/11/23-14:59:09.800228 139957668369984 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-14:59:09.800493 139957668369984 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-14:59:09.862208 139957668369984 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28617/att-00/MANIFEST-206347 2024/11/23-14:59:09.878469 139957668369984 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28617/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 265029, last_sequence is 4047160318, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 264884 2024/11/23-14:59:09.878477 139957668369984 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-14:59:09.878607 139957668369984 [version_set.cc:5721] Creating manifest 265029 2024/11/23-14:59:09.911938 139957668369984 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-14:59:09.936939 139957668369984 [compaction_job.cc:2437] [default] [JOB 28617] Compacting 1@2 + 20@3 files to L3, score 1.00, subcompactions 1 : 1 2024/11/23-14:59:09.936962 139957668369984 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 21380 Base level 2, inputs: [265019(31MB)], [264942(40MB) 264943(40MB) 264944(40MB) 264945(40MB) 264946(40MB) 264947(40MB) 264948(40MB) 264949(40MB) 264950(40MB) 264951(40MB) 264952(40MB) 264953(40MB) 264954(40MB) 264955(40MB) 264956(40MB) 264957(40MB) 264958(40MB) 264959(40MB) 264960(39MB) 264963(40MB)] 2024/11/23-14:59:09.944508 139957668369984 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:59:09.944551 139957668369984 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:59:09.944580 139957668369984 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-14:59:09.944588 139957668369984 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:59:09.944603 139957668369984 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-14:59:10.678397 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265030.sst) /dev/shm/ToplingTemp/Topling-84680-wypHRg 2024/11/23-14:59:11.014508 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265030.sst): first pass time = 0.34's, 399.390'MB/sec 2024/11/23-14:59:12.070708 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265030.sst): old prealloc_size = 147639500, real_size = 40671440 2024/11/23-14:59:12.070855 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265030.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.81 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.99 avg-zval = 312.18 2024/11/23-14:59:12.071061 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265030.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:12.337671 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265030.sst): second pass time = 1.05's, 124.909'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.27's, 159.230'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1861.034'MB/sec (index lex order gen) rebuild zvType time = 0.20's, 0.165'MB/sec write SST data time = 0.07's, 599.137'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48280.848'MB/sec zip my value time = 1.05's, unzip length = 0.132'GB zip my value throughput = 124.909'MB/sec zip pipeline throughput = 150.889'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.81 avg-val = 1016.99 avg-zval = 312.18 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.81 avg-val = 1023.99 avg-zval = 312.18 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6470 value = 3.2577 dict = 1.00 all = 3.1578 } Zip/UnZip{ index = 0.1504 value = 0.3070 dict = 1.00 all = 0.3167 } ---------------------------- total value len = 8.981156 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679866 Billion total entry num = 0.052487595 Billion write speed all = 0.477312584 MB/sec (with seq num) write speed all = 0.458086467 MB/sec (without seq num) 2024/11/23-14:59:12.393270 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265030: 129431 keys, 42503901 bytes, temperature: kWarm 2024/11/23-14:59:12.393382 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345152393308, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265030, "file_size": 42503901, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4047121685, "table_properties": {"data_size": 40405248, "index_size": 233664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131630025, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345150, "oldest_key_time": 0, "file_creation_time": 1732345150, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265030, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:12.459029 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265031.sst) /dev/shm/ToplingTemp/Topling-84680-LueDhk 2024/11/23-14:59:13.336918 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265031.sst): first pass time = 0.88's, 152.895'MB/sec 2024/11/23-14:59:15.041462 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265031.sst): old prealloc_size = 147639500, real_size = 40826592 2024/11/23-14:59:15.041579 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265031.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.98 avg-zval = 313.42 2024/11/23-14:59:15.041756 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265031.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:15.231009 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265031.sst): second pass time = 1.70's, 77.306'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 225.089'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2341.720'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.230'MB/sec write SST data time = 0.05's, 870.140'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 37424.431'MB/sec zip my value time = 1.70's, unzip length = 0.132'GB zip my value throughput = 77.306'MB/sec zip pipeline throughput = 94.903'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1016.98 avg-zval = 313.42 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1023.98 avg-zval = 313.42 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8112 value = 3.2448 dict = 1.00 all = 3.1463 } Zip/UnZip{ index = 0.1468 value = 0.3082 dict = 1.00 all = 0.3178 } ---------------------------- total value len = 9.113691 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809297 Billion total entry num = 0.052617026 Billion write speed all = 0.483435490 MB/sec (with seq num) write speed all = 0.464164516 MB/sec (without seq num) 2024/11/23-14:59:15.288551 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265031: 129431 keys, 42659086 bytes, temperature: kWarm 2024/11/23-14:59:15.288614 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345155288577, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265031, "file_size": 42659086, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4047119293, "table_properties": {"data_size": 40566032, "index_size": 228032, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629148, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345152, "oldest_key_time": 0, "file_creation_time": 1732345152, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265031, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:15.330492 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265032.sst) /dev/shm/ToplingTemp/Topling-84680-8gYbWi 2024/11/23-14:59:15.989551 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265032.sst): first pass time = 0.66's, 203.666'MB/sec 2024/11/23-14:59:16.844870 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265032.sst): old prealloc_size = 147639500, real_size = 40682128 2024/11/23-14:59:16.845016 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265032.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1016.97 avg-zval = 312.26 2024/11/23-14:59:16.845174 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265032.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:17.019547 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265032.sst): second pass time = 0.85's, 154.021'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 243.413'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1923.685'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.262'MB/sec write SST data time = 0.05's, 836.747'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45486.217'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 154.021'MB/sec zip pipeline throughput = 154.021'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1016.97 avg-zval = 312.26 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1023.97 avg-zval = 312.26 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0425 GB } UnZip/Zip{ index = 6.6599 value = 3.2568 dict = 1.00 all = 3.1570 } Zip/UnZip{ index = 0.1502 value = 0.3070 dict = 1.00 all = 0.3168 } ---------------------------- total value len = 9.246227 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938730 Billion total entry num = 0.052746459 Billion write speed all = 0.489581572 MB/sec (with seq num) write speed all = 0.470264775 MB/sec (without seq num) 2024/11/23-14:59:17.074370 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265032: 129433 keys, 42514718 bytes, temperature: kWarm 2024/11/23-14:59:17.074432 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345157074401, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265032, "file_size": 42514718, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237870807, "largest_seqno": 4047123142, "table_properties": {"data_size": 40416384, "index_size": 233216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629915, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345155, "oldest_key_time": 0, "file_creation_time": 1732345155, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265032, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:17.116309 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265033.sst) /dev/shm/ToplingTemp/Topling-84680-qrTmKj 2024/11/23-14:59:17.961742 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265033.sst): first pass time = 0.85's, 158.764'MB/sec 2024/11/23-14:59:19.713355 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265033.sst): old prealloc_size = 147639500, real_size = 40812880 2024/11/23-14:59:19.713502 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265033.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129435 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.95 avg-zval = 313.30 2024/11/23-14:59:19.713701 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265033.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:19.925066 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265033.sst): second pass time = 1.74's, 75.441'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 201.457'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1850.346'MB/sec (index lex order gen) rebuild zvType time = 0.16's, 0.200'MB/sec write SST data time = 0.05's, 851.373'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42543.770'MB/sec zip my value time = 1.74's, unzip length = 0.132'GB zip my value throughput = 75.441'MB/sec zip pipeline throughput = 119.876'MB/sec entries = 129435 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1016.95 avg-zval = 313.30 usrkeys = 129435 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1023.95 avg-zval = 313.30 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129435 bytes = 485392 } seq expand size = 906045 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.8000 value = 3.2459 dict = 1.00 all = 3.1473 } Zip/UnZip{ index = 0.1471 value = 0.3081 dict = 1.00 all = 0.3177 } ---------------------------- total value len = 9.378762 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068165 Billion total entry num = 0.052875894 Billion write speed all = 0.495701267 MB/sec (with seq num) write speed all = 0.476339643 MB/sec (without seq num) 2024/11/23-14:59:19.978201 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265033: 129435 keys, 42645427 bytes, temperature: kWarm 2024/11/23-14:59:19.978281 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345159978244, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265033, "file_size": 42645427, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4047113076, "table_properties": {"data_size": 40551936, "index_size": 228416, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588700, "raw_average_key_size": 20, "raw_value_size": 131629068, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129435, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345157, "oldest_key_time": 0, "file_creation_time": 1732345157, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265033, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:20.021797 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265034.sst) /dev/shm/ToplingTemp/Topling-84680-Bun7Dg 2024/11/23-14:59:20.707021 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265034.sst): first pass time = 0.69's, 195.887'MB/sec 2024/11/23-14:59:21.573149 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265034.sst): old prealloc_size = 147639500, real_size = 40588752 2024/11/23-14:59:21.573295 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265034.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.52 2024/11/23-14:59:21.573492 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265034.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:21.757275 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265034.sst): second pass time = 0.86's, 152.255'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 230.414'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1913.487'MB/sec (index lex order gen) rebuild zvType time = 0.13's, 0.241'MB/sec write SST data time = 0.05's, 849.394'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 34814.701'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 152.255'MB/sec zip pipeline throughput = 152.255'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1017.00 avg-zval = 311.52 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1024.00 avg-zval = 311.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5874 value = 3.2646 dict = 1.00 all = 3.1639 } Zip/UnZip{ index = 0.1518 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.511298 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197595 Billion total entry num = 0.053005324 Billion write speed all = 0.501844016 MB/sec (with seq num) write speed all = 0.482436626 MB/sec (without seq num) 2024/11/23-14:59:21.809812 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265034: 129430 keys, 42421833 bytes, temperature: kWarm 2024/11/23-14:59:21.809879 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345161809845, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265034, "file_size": 42421833, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4047121351, "table_properties": {"data_size": 40320448, "index_size": 235776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629871, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345160, "oldest_key_time": 0, "file_creation_time": 1732345160, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265034, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:21.866651 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265035.sst) /dev/shm/ToplingTemp/Topling-84680-r7v8ei 2024/11/23-14:59:22.962810 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265035.sst): first pass time = 1.10's, 122.450'MB/sec 2024/11/23-14:59:24.280484 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265035.sst): old prealloc_size = 147639500, real_size = 40900608 2024/11/23-14:59:24.280629 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265035.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1017.01 avg-zval = 313.99 2024/11/23-14:59:24.280832 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265035.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:24.503707 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265035.sst): second pass time = 1.32's, 99.966'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.22's, 191.445'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1710.429'MB/sec (index lex order gen) rebuild zvType time = 0.18's, 0.184'MB/sec write SST data time = 0.05's, 912.804'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44638.195'MB/sec zip my value time = 1.32's, unzip length = 0.132'GB zip my value throughput = 99.966'MB/sec zip pipeline throughput = 126.638'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1017.01 avg-zval = 313.99 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1024.01 avg-zval = 313.99 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.7863 value = 3.2390 dict = 1.00 all = 3.1408 } Zip/UnZip{ index = 0.1474 value = 0.3087 dict = 1.00 all = 0.3184 } ---------------------------- total value len = 9.643834 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327024 Billion total entry num = 0.053134753 Billion write speed all = 0.507964491 MB/sec (with seq num) write speed all = 0.488512157 MB/sec (without seq num) 2024/11/23-14:59:24.556647 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265035: 129429 keys, 42733506 bytes, temperature: kWarm 2024/11/23-14:59:24.556715 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345164556679, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265035, "file_size": 42733506, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237885133, "largest_seqno": 4047115909, "table_properties": {"data_size": 40639216, "index_size": 228864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630083, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345161, "oldest_key_time": 0, "file_creation_time": 1732345161, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265035, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:24.595207 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265036.sst) /dev/shm/ToplingTemp/Topling-84680-hjAgGg 2024/11/23-14:59:25.159703 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265036.sst): first pass time = 0.56's, 237.782'MB/sec 2024/11/23-14:59:26.322658 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265036.sst): old prealloc_size = 147639500, real_size = 41004128 2024/11/23-14:59:26.322771 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265036.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.03 avg-zval = 314.83 2024/11/23-14:59:26.322930 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265036.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:26.487306 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265036.sst): second pass time = 1.16's, 113.257'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 260.190'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2444.925'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.04's, 985.391'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44126.705'MB/sec zip my value time = 1.16's, unzip length = 0.132'GB zip my value throughput = 113.257'MB/sec zip pipeline throughput = 132.187'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.03 avg-zval = 314.83 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.03 avg-zval = 314.83 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129426 bytes = 485360 } seq expand size = 905982 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9375 value = 3.2304 dict = 1.00 all = 3.1332 } Zip/UnZip{ index = 0.1441 value = 0.3096 dict = 1.00 all = 0.3192 } ---------------------------- total value len = 9.776370 GB avg = 0.184 KB (by entry num) total key len = 1.458941 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456450 Billion total entry num = 0.053264179 Billion write speed all = 0.514101346 MB/sec (with seq num) write speed all = 0.494603399 MB/sec (without seq num) 2024/11/23-14:59:26.540763 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265036: 129426 keys, 42836892 bytes, temperature: kWarm 2024/11/23-14:59:26.540830 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345166540796, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265036, "file_size": 42836892, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4047054427, "table_properties": {"data_size": 40747728, "index_size": 223872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629591, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345164, "oldest_key_time": 0, "file_creation_time": 1732345164, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265036, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:26.582692 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265037.sst) /dev/shm/ToplingTemp/Topling-84680-t288wk 2024/11/23-14:59:27.253327 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265037.sst): first pass time = 0.67's, 200.148'MB/sec 2024/11/23-14:59:28.099669 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265037.sst): old prealloc_size = 147639500, real_size = 40939120 2024/11/23-14:59:28.099816 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265037.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129434 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1016.96 avg-zval = 314.33 2024/11/23-14:59:28.100009 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265037.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:28.312694 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265037.sst): second pass time = 0.85's, 155.707'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.21's, 200.785'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1773.937'MB/sec (index lex order gen) rebuild zvType time = 0.17's, 0.193'MB/sec write SST data time = 0.04's, 952.584'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 33746.295'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 155.707'MB/sec zip pipeline throughput = 164.777'MB/sec entries = 129434 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.96 avg-zval = 314.33 usrkeys = 129434 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.96 avg-zval = 314.33 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129434 bytes = 485392 } seq expand size = 906038 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 7.0060 value = 3.2353 dict = 1.00 all = 3.1380 } Zip/UnZip{ index = 0.1427 value = 0.3091 dict = 1.00 all = 0.3187 } ---------------------------- total value len = 9.908905 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585884 Billion total entry num = 0.053393613 Billion write speed all = 0.520240844 MB/sec (with seq num) write speed all = 0.500697149 MB/sec (without seq num) 2024/11/23-14:59:28.364066 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265037: 129434 keys, 42771544 bytes, temperature: kWarm 2024/11/23-14:59:28.364137 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345168364102, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265037, "file_size": 42771544, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874784, "largest_seqno": 4047123372, "table_properties": {"data_size": 40684896, "index_size": 221696, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588680, "raw_average_key_size": 20, "raw_value_size": 131629190, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129434, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345166, "oldest_key_time": 0, "file_creation_time": 1732345166, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265037, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:28.399274 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265038.sst) /dev/shm/ToplingTemp/Topling-84680-I83PHg 2024/11/23-14:59:28.897248 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265038.sst): first pass time = 0.50's, 269.548'MB/sec 2024/11/23-14:59:29.671818 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265038.sst): old prealloc_size = 147639500, real_size = 40594784 2024/11/23-14:59:29.671936 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265038.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.57 2024/11/23-14:59:29.672105 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265038.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:29.833729 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265038.sst): second pass time = 0.77's, 170.087'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.070'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2413.809'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.04's, 994.702'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 50379.800'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 170.087'MB/sec zip pipeline throughput = 170.087'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1016.99 avg-zval = 311.57 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1023.99 avg-zval = 311.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5875 value = 3.2641 dict = 1.00 all = 3.1635 } Zip/UnZip{ index = 0.1518 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.041440 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715315 Billion total entry num = 0.053523044 Billion write speed all = 0.526386655 MB/sec (with seq num) write speed all = 0.506796948 MB/sec (without seq num) 2024/11/23-14:59:29.885020 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265038: 129431 keys, 42427221 bytes, temperature: kWarm 2024/11/23-14:59:29.885085 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345169885051, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265038, "file_size": 42427221, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873877, "largest_seqno": 4047122783, "table_properties": {"data_size": 40326480, "index_size": 235776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629473, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345168, "oldest_key_time": 0, "file_creation_time": 1732345168, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265038, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:29.925490 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265039.sst) /dev/shm/ToplingTemp/Topling-84680-SHI1nj 2024/11/23-14:59:30.435402 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265039.sst): first pass time = 0.51's, 263.238'MB/sec 2024/11/23-14:59:31.220968 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265039.sst): old prealloc_size = 147639500, real_size = 40595024 2024/11/23-14:59:31.221101 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265039.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.82 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.57 2024/11/23-14:59:31.221251 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265039.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:31.380001 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265039.sst): second pass time = 0.78's, 167.709'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 266.827'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2369.322'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 1005.002'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45159.483'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 167.709'MB/sec zip pipeline throughput = 167.709'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.82 avg-val = 1016.99 avg-zval = 311.57 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.82 avg-val = 1023.99 avg-zval = 311.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.5874 value = 3.2641 dict = 1.00 all = 3.1635 } Zip/UnZip{ index = 0.1518 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.173976 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844745 Billion total entry num = 0.053652474 Billion write speed all = 0.532530992 MB/sec (with seq num) write speed all = 0.512895302 MB/sec (without seq num) 2024/11/23-14:59:31.432025 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265039: 129430 keys, 42427652 bytes, temperature: kWarm 2024/11/23-14:59:31.432094 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345171432058, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265039, "file_size": 42427652, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4047117114, "table_properties": {"data_size": 40326720, "index_size": 235776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629397, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345169, "oldest_key_time": 0, "file_creation_time": 1732345169, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265039, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:31.467725 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265040.sst) /dev/shm/ToplingTemp/Topling-84680-fhFcok 2024/11/23-14:59:32.199852 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265040.sst): first pass time = 0.73's, 183.335'MB/sec 2024/11/23-14:59:33.031092 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265040.sst): old prealloc_size = 147639500, real_size = 41011664 2024/11/23-14:59:33.031206 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265040.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.99 avg-zval = 314.87 2024/11/23-14:59:33.031361 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265040.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:33.189785 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265040.sst): second pass time = 0.83's, 158.495'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 270.020'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2414.073'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 1025.122'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49486.749'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 158.495'MB/sec zip pipeline throughput = 158.495'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.99 avg-zval = 314.87 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.99 avg-zval = 314.87 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.8885 value = 3.2299 dict = 1.00 all = 3.1327 } Zip/UnZip{ index = 0.1452 value = 0.3096 dict = 1.00 all = 0.3192 } ---------------------------- total value len = 10.306511 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974175 Billion total entry num = 0.053781904 Billion write speed all = 0.538667961 MB/sec (with seq num) write speed all = 0.518986532 MB/sec (without seq num) 2024/11/23-14:59:33.241213 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265040: 129430 keys, 42844439 bytes, temperature: kWarm 2024/11/23-14:59:33.241275 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345173241243, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265040, "file_size": 42844439, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237888398, "largest_seqno": 4047103687, "table_properties": {"data_size": 40753664, "index_size": 225472, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629294, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345171, "oldest_key_time": 0, "file_creation_time": 1732345171, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265040, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:33.280083 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265041.sst) /dev/shm/ToplingTemp/Topling-84680-63zG9k 2024/11/23-14:59:33.894741 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265041.sst): first pass time = 0.61's, 218.378'MB/sec 2024/11/23-14:59:34.725291 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265041.sst): old prealloc_size = 147639500, real_size = 40719744 2024/11/23-14:59:34.725415 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265041.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.80 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.04 avg-zval = 312.57 2024/11/23-14:59:34.726077 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265041.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:34.885881 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265041.sst): second pass time = 0.83's, 158.615'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 265.011'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2316.400'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 996.144'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48516.124'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 158.615'MB/sec zip pipeline throughput = 158.615'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.80 avg-val = 1017.04 avg-zval = 312.57 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.80 avg-val = 1024.04 avg-zval = 312.57 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129425 bytes = 485360 } seq expand size = 905975 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.6577 value = 3.2538 dict = 1.00 all = 3.1542 } Zip/UnZip{ index = 0.1502 value = 0.3073 dict = 1.00 all = 0.3170 } ---------------------------- total value len = 10.439047 GB avg = 0.194 KB (by entry num) total key len = 1.471884 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103600 Billion total entry num = 0.053911329 Billion write speed all = 0.544806779 MB/sec (with seq num) write speed all = 0.525079517 MB/sec (without seq num) 2024/11/23-14:59:34.937081 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265041: 129425 keys, 42552515 bytes, temperature: kWarm 2024/11/23-14:59:34.937152 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345174937116, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265041, "file_size": 42552515, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237871365, "largest_seqno": 4047114184, "table_properties": {"data_size": 40453936, "index_size": 233280, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131630104, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345173, "oldest_key_time": 0, "file_creation_time": 1732345173, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265041, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:34.977229 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265042.sst) /dev/shm/ToplingTemp/Topling-84680-q2Wy5k 2024/11/23-14:59:35.537653 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265042.sst): first pass time = 0.56's, 239.511'MB/sec 2024/11/23-14:59:36.322689 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265042.sst): old prealloc_size = 147639500, real_size = 40480752 2024/11/23-14:59:36.322813 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265042.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.79 2024/11/23-14:59:36.322923 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265042.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:36.483296 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265042.sst): second pass time = 0.78's, 168.014'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.468'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2239.335'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 972.492'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49983.893'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 168.014'MB/sec zip pipeline throughput = 168.014'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 310.79 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1019.00 avg-zval = 310.79 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 37132 bytes = 139264 } seq expand size = 259924 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9636 value = 3.2723 dict = 1.00 all = 3.1966 } Zip/UnZip{ index = 0.1436 value = 0.3056 dict = 1.00 all = 0.3128 } ---------------------------- total value len = 10.570936 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233030 Billion total entry num = 0.054040759 Billion write speed all = 0.550917552 MB/sec (with seq num) write speed all = 0.531144374 MB/sec (without seq num) 2024/11/23-14:59:36.535243 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265042: 129430 keys, 41987533 bytes, temperature: kWarm 2024/11/23-14:59:36.535313 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345176535277, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265042, "file_size": 41987533, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4047114985, "table_properties": {"data_size": 40225184, "index_size": 223040, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629354, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345174, "oldest_key_time": 0, "file_creation_time": 1732345174, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265042, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:36.572248 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265043.sst) /dev/shm/ToplingTemp/Topling-84680-0Boa6g 2024/11/23-14:59:37.302748 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265043.sst): first pass time = 0.73's, 183.743'MB/sec 2024/11/23-14:59:38.142511 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265043.sst): old prealloc_size = 147639500, real_size = 41070080 2024/11/23-14:59:38.142645 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265043.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.99 avg-zval = 315.35 2024/11/23-14:59:38.142822 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265043.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:38.308714 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265043.sst): second pass time = 0.84's, 156.883'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 258.047'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1973.372'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.266'MB/sec write SST data time = 0.04's, 967.415'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44009.391'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 156.883'MB/sec zip pipeline throughput = 156.883'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.99 avg-zval = 315.35 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.40 avg-zval = 315.35 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 118523 bytes = 444480 } seq expand size = 829661 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 7.0139 value = 3.2249 dict = 1.00 all = 3.1299 } Zip/UnZip{ index = 0.1426 value = 0.3101 dict = 1.00 all = 0.3195 } ---------------------------- total value len = 10.703395 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362460 Billion total entry num = 0.054170189 Billion write speed all = 0.557047679 MB/sec (with seq num) write speed all = 0.537228798 MB/sec (without seq num) 2024/11/23-14:59:38.362701 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265043: 129430 keys, 42882013 bytes, temperature: kWarm 2024/11/23-14:59:38.362772 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345178362737, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265043, "file_size": 42882013, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4047103173, "table_properties": {"data_size": 40816112, "index_size": 221440, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629323, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345176, "oldest_key_time": 0, "file_creation_time": 1732345176, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265043, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:38.399904 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265044.sst) /dev/shm/ToplingTemp/Topling-84680-E27ujh 2024/11/23-14:59:39.229566 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265044.sst): first pass time = 0.83's, 161.783'MB/sec 2024/11/23-14:59:40.066232 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265044.sst): old prealloc_size = 147639500, real_size = 40713920 2024/11/23-14:59:40.066362 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265044.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.01 avg-zval = 312.57 2024/11/23-14:59:40.066485 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265044.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:40.230408 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265044.sst): second pass time = 0.84's, 157.457'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.133'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2101.450'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.05's, 940.337'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48728.932'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 157.457'MB/sec zip pipeline throughput = 157.457'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1017.01 avg-zval = 312.57 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1021.22 avg-zval = 312.57 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 77890 bytes = 292096 } seq expand size = 545230 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.8864 value = 3.2537 dict = 1.00 all = 3.1675 } Zip/UnZip{ index = 0.1452 value = 0.3073 dict = 1.00 all = 0.3157 } ---------------------------- total value len = 10.835570 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021175 GB avg = 0.015 KB total ukey num = 0.067491888 Billion total entry num = 0.054299617 Billion write speed all = 0.563161290 MB/sec (with seq num) write speed all = 0.543296802 MB/sec (without seq num) 2024/11/23-14:59:40.282221 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265044: 129428 keys, 42373619 bytes, temperature: kWarm 2024/11/23-14:59:40.282289 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345180282255, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265044, "file_size": 42373619, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4047117115, "table_properties": {"data_size": 40455856, "index_size": 225536, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629217, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345178, "oldest_key_time": 0, "file_creation_time": 1732345178, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265044, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:40.318993 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265045.sst) /dev/shm/ToplingTemp/Topling-84680-lz0Z5g 2024/11/23-14:59:40.905110 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265045.sst): first pass time = 0.59's, 229.015'MB/sec 2024/11/23-14:59:41.759718 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265045.sst): old prealloc_size = 147639500, real_size = 40957952 2024/11/23-14:59:41.759841 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265045.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.02 avg-zval = 314.46 2024/11/23-14:59:41.760009 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265045.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:41.921686 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265045.sst): second pass time = 0.85's, 154.152'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.064'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2230.234'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 969.581'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43501.673'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 154.152'MB/sec zip pipeline throughput = 154.152'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1017.02 avg-zval = 314.46 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.30 avg-zval = 314.46 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 116137 bytes = 435536 } seq expand size = 812959 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.8825 value = 3.2342 dict = 1.00 all = 3.1387 } Zip/UnZip{ index = 0.1453 value = 0.3092 dict = 1.00 all = 0.3186 } ---------------------------- total value len = 10.968013 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621315 Billion total entry num = 0.054429044 Billion write speed all = 0.569292092 MB/sec (with seq num) write speed all = 0.549381795 MB/sec (without seq num) 2024/11/23-14:59:41.972196 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265045: 129427 keys, 42761776 bytes, temperature: kWarm 2024/11/23-14:59:41.972264 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345181972228, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265045, "file_size": 42761776, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4047117679, "table_properties": {"data_size": 40699760, "index_size": 225664, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629803, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345180, "oldest_key_time": 0, "file_creation_time": 1732345180, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265045, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:42.012939 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265046.sst) /dev/shm/ToplingTemp/Topling-84680-8gV5lj 2024/11/23-14:59:42.816789 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265046.sst): first pass time = 0.80's, 166.977'MB/sec 2024/11/23-14:59:43.681759 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265046.sst): old prealloc_size = 147639500, real_size = 40503904 2024/11/23-14:59:43.681878 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265046.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.95 2024/11/23-14:59:43.681991 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265046.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:43.843766 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265046.sst): second pass time = 0.86's, 152.288'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.619'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2247.194'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 973.922'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47602.999'MB/sec zip my value time = 0.86's, unzip length = 0.132'GB zip my value throughput = 152.288'MB/sec zip pipeline throughput = 152.288'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.99 avg-zval = 310.95 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1019.64 avg-zval = 310.95 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 48914 bytes = 183440 } seq expand size = 342398 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.8963 value = 3.2706 dict = 1.00 all = 3.1915 } Zip/UnZip{ index = 0.1450 value = 0.3058 dict = 1.00 all = 0.3133 } ---------------------------- total value len = 11.099984 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750745 Billion total entry num = 0.054558474 Billion write speed all = 0.575394335 MB/sec (with seq num) write speed all = 0.555438447 MB/sec (without seq num) 2024/11/23-14:59:43.896109 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265046: 129430 keys, 42054955 bytes, temperature: kWarm 2024/11/23-14:59:43.896174 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345183896141, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265046, "file_size": 42054955, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4047082377, "table_properties": {"data_size": 40246160, "index_size": 225216, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629241, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345181, "oldest_key_time": 0, "file_creation_time": 1732345181, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265046, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:43.936053 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265047.sst) /dev/shm/ToplingTemp/Topling-84680-TJnQ2i 2024/11/23-14:59:44.433616 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265047.sst): first pass time = 0.50's, 269.770'MB/sec 2024/11/23-14:59:45.274394 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265047.sst): old prealloc_size = 147639500, real_size = 40588896 2024/11/23-14:59:45.274514 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265047.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129425 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.04 avg-zval = 311.64 2024/11/23-14:59:45.274601 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265047.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:45.436678 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265047.sst): second pass time = 0.84's, 156.843'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.203'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2257.094'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 949.368'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42008.015'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 156.843'MB/sec zip pipeline throughput = 156.843'MB/sec entries = 129425 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.04 avg-zval = 311.64 usrkeys = 129425 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.48 avg-zval = 311.64 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 26719 bytes = 100208 } seq expand size = 187033 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9934 value = 3.2635 dict = 1.00 all = 3.1913 } Zip/UnZip{ index = 0.1430 value = 0.3064 dict = 1.00 all = 0.3133 } ---------------------------- total value len = 11.231801 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880170 Billion total entry num = 0.054687899 Billion write speed all = 0.581497179 MB/sec (with seq num) write speed all = 0.561495408 MB/sec (without seq num) 2024/11/23-14:59:45.489815 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265047: 129425 keys, 42057150 bytes, temperature: kWarm 2024/11/23-14:59:45.489881 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345185489849, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265047, "file_size": 42057150, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4047111481, "table_properties": {"data_size": 40334288, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588500, "raw_average_key_size": 20, "raw_value_size": 131629880, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129425, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345183, "oldest_key_time": 0, "file_creation_time": 1732345183, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265047, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:45.537216 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265048.sst) /dev/shm/ToplingTemp/Topling-84680-y2CKRk 2024/11/23-14:59:46.323919 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265048.sst): first pass time = 0.79's, 170.618'MB/sec 2024/11/23-14:59:47.211735 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265048.sst): old prealloc_size = 147639500, real_size = 40453280 2024/11/23-14:59:47.211862 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265048.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.58 2024/11/23-14:59:47.211946 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265048.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:47.374198 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265048.sst): second pass time = 0.89's, 148.538'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 258.060'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2186.106'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.04's, 990.967'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 38016.196'MB/sec zip my value time = 0.89's, unzip length = 0.132'GB zip my value throughput = 148.538'MB/sec zip pipeline throughput = 148.538'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 310.58 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.41 avg-zval = 310.58 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 26143 bytes = 98048 } seq expand size = 183001 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0419 GB } UnZip/Zip{ index = 6.9937 value = 3.2745 dict = 1.00 all = 3.2019 } Zip/UnZip{ index = 0.1430 value = 0.3054 dict = 1.00 all = 0.3123 } ---------------------------- total value len = 11.363614 GB avg = 0.207 KB (by entry num) total key len = 1.490004 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009600 Billion total entry num = 0.054817329 Billion write speed all = 0.587589676 MB/sec (with seq num) write speed all = 0.567542342 MB/sec (without seq num) 2024/11/23-14:59:47.424816 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265048: 129430 keys, 41918850 bytes, temperature: kWarm 2024/11/23-14:59:47.424880 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345187424847, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265048, "file_size": 41918850, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4047123092, "table_properties": {"data_size": 40198672, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629403, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345185, "oldest_key_time": 0, "file_creation_time": 1732345185, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265048, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:47.465326 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265049.sst) /dev/shm/ToplingTemp/Topling-84680-LoeZaj 2024/11/23-14:59:48.030837 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265049.sst): first pass time = 0.57's, 237.355'MB/sec 2024/11/23-14:59:48.848685 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265049.sst): old prealloc_size = 147639500, real_size = 40646144 2024/11/23-14:59:48.848809 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265049.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.00 avg-zval = 312.07 2024/11/23-14:59:48.848919 139957668369984 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265049.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-14:59:49.010161 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265049.sst): second pass time = 0.82's, 161.080'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.591'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2174.079'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 980.176'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49260.079'MB/sec zip my value time = 0.82's, unzip length = 0.132'GB zip my value throughput = 161.080'MB/sec zip pipeline throughput = 161.080'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.00 avg-zval = 312.07 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1020.22 avg-zval = 312.07 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 59514 bytes = 223200 } seq expand size = 416598 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 6.9936 value = 3.2588 dict = 1.00 all = 3.1777 } Zip/UnZip{ index = 0.1430 value = 0.3069 dict = 1.00 all = 0.3147 } ---------------------------- total value len = 11.495660 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139029 Billion total entry num = 0.054946758 Billion write speed all = 0.593699969 MB/sec (with seq num) write speed all = 0.573606804 MB/sec (without seq num) 2024/11/23-14:59:49.061217 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265049: 129429 keys, 42237368 bytes, temperature: kWarm 2024/11/23-14:59:49.061289 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345189061252, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265049, "file_size": 42237368, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4047123119, "table_properties": {"data_size": 40391536, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131629725, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732345187, "oldest_key_time": 0, "file_creation_time": 1732345187, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265049, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:49.099157 139957668369984 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265050.sst) /dev/shm/ToplingTemp/Topling-84680-KzzNyj 2024/11/23-14:59:49.108704 139957668369984 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265050.sst): first pass time = 0.01's, 402.096'MB/sec 2024/11/23-14:59:49.142731 139957668369984 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265050.sst): old prealloc_size = 147639500, real_size = 1203792 2024/11/23-14:59:49.142779 139957668369984 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265050.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 3687 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.24 raw-val = 0.0038 GB zip-val = 0.0012 GB avg-val =1017.12 avg-zval = 325.96 2024/11/23-14:59:49.149334 139957668369984 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00/265050.sst): second pass time = 0.03's, 112.038'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.01's, 190.578'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 31.678'MB/sec (index lex order gen) rebuild zvType time = 0.01's, 0.178'MB/sec write SST data time = 0.00's, 907.866'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.042'MB, throughput = 1892.531'MB/sec zip my value time = 0.03's, unzip length = 0.004'GB zip my value throughput = 112.038'MB/sec zip pipeline throughput = 112.038'MB/sec entries = 3687 avg-key = 20.00 avg-zkey = 0.24 avg-val = 1017.12 avg-zval = 325.96 usrkeys = 3687 avg-key = 12.00 avg-zkey = 0.24 avg-val = 1022.52 avg-zval = 325.96 TagRS{ kinds = RS_Key0_TagN bytes = 680 } TagArray{ size = 2845 bytes = 10688 } seq expand size = 19915 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0000 GB value = 0.0038 GB dict = 0.04 MB all = 0.0038 GB } __ZipSize{ index = 0.0000 GB value = 0.0012 GB dict = 0.04 MB all = 0.0013 GB } UnZip/Zip{ index = 49.3795 value = 3.1204 dict = 1.00 all = 3.0423 } Zip/UnZip{ index = 0.0203 value = 0.3205 dict = 1.00 all = 0.3287 } ---------------------------- total value len = 11.499430 GB avg = 0.209 KB (by entry num) total key len = 1.492667 GB avg = 0.027 KB total ukey len = 1.028985 GB avg = 0.015 KB total ukey num = 0.068142716 Billion total entry num = 0.054950445 Billion write speed all = 0.593871892 MB/sec (with seq num) write speed all = 0.573777507 MB/sec (without seq num) 2024/11/23-14:59:49.157326 139957668369984 [compaction_job.cc:1948] [default] [JOB 28617] Generated table #265050: 3687 keys, 1256916 bytes, temperature: kWarm 2024/11/23-14:59:49.157372 139957668369984 EVENT_LOG_v1 {"time_micros": 1732345189157346, "cf_name": "default", "job": 28617, "event": "table_file_creation", "file_number": 265050, "file_size": 1256916, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4045619232, "table_properties": {"data_size": 1201808, "index_size": 896, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 73740, "raw_average_key_size": 20, "raw_value_size": 3750133, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 3687, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1732345189, "oldest_key_time": 0, "file_creation_time": 1732345189, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 265050, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-14:59:49.165157 139957668369984 [compaction_job.cc:703] job-28617: subcompact[0], size: 0.850869 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-28617/att-00 [265030,265031,265032,265033,265034,265035,265036,265037,265038,265039,265040,265041,265042,265043,265044,265045,265046,265047,265048,265049,265050] 2024/11/23-14:59:49.293133 139957668369984 (Original Log Time 2024/11/23-14:59:09.937069) EVENT_LOG_v1 {"time_micros": 1732345149936998, "job": 28617, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [265019], "files_L3": [264942, 264943, 264944, 264945, 264946, 264947, 264948, 264949, 264950, 264951, 264952, 264953, 264954, 264955, 264956, 264957, 264958, 264959, 264960, 264963], "score": 0.997776, "input_data_size": 882305804, "oldest_snapshot_seqno": 4047160318} 2024/11/23-14:59:49.293136 139957668369984 (Original Log Time 2024/11/23-14:59:49.291025) [compaction_job.cc:2022] [default] [JOB 28617] Compacted 1@2 + 20@3 files to L3 => 850868914 bytes 2024/11/23-14:59:49.293138 139957668369984 (Original Log Time 2024/11/23-14:59:49.293062) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 21 0 0 0] max score 0.03, MB/sec: 22.5 rd, 21.7 wr, level 3, files in(1, 20) out(21 +0 blob) MB in(31.2, 810.2 +0.0 blob) out(811.5 +0.0 blob), read-write-amplify(53.0) write-amplify(26.0) OK, records in: 2620503, records dropped: 28222 output_compression: Snappy 2024/11/23-14:59:49.293141 139957668369984 (Original Log Time 2024/11/23-14:59:49.293107) EVENT_LOG_v1 {"time_micros": 1732345189293086, "cf": "default", "job": 28617, "event": "compaction_finished", "compaction_time_micros": 39228090, "compaction_time_cpu_micros": 8088439, "output_level": 3, "num_output_files": 21, "total_output_size": 850868914, "num_input_records": 2620503, "num_output_records": 2592281, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 616026627, "file_range_sync_nanos": 0, "file_fsync_nanos": 15145, "file_prepare_write_nanos": 57681, "lsm_state": [0, 0, 0, 21, 0, 0, 0]} 2024/11/23-14:59:49.427758 139957668369984 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732345149,"executesMs":39492,"compactionJobId":28617,"attempt":0,"compactionInputRawBytes":2717450792,"compactionInputZipBytes":882305804,"compactionOutputRawBytes":0,"compactionOutputZipBytes":850868914,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-8861860d3783775a838a298e439d7bf0-9318ee7d856b419c-00"}}, response = 2024/11/23-14:59:49.427851 139957668369984 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-28617/att-00: olev 3, work 39.493 s, result 7.781 ms, install 2.134 ms, input{raw 2.531 GiB zip 841.432 MiB}