2024/11/23-20:14:00.276256 139957634799168 DEBG dcompact_worker.cpp:2214: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00: fork to child time = 0.010225 sec 2024/11/23-20:14:00.277048 139957634799168 DEBG dcompact_worker.cpp:1013: Beg SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00 2024/11/23-20:14:00.285267 139957634799168 DEBG dcompact_worker.cpp:1033: End SerDeRead: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00 2024/11/23-20:14:00.286148 139957634799168 TRAC dcompact_worker.cpp:1083: INFO_LOG_LEVEL: rpc = INFO_LEVEL, env_var = undefined 2024/11/23-20:14:00.286451 139957634799168 [WARN] [column_family.cc:386] multiple cf_paths/db_paths and level_compaction_dynamic_level_bytes can't be used together 2024/11/23-20:14:00.342438 139957634799168 [version_set.cc:6208] Recovering from manifest file: /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34954/att-00/MANIFEST-206347 2024/11/23-20:14:00.374572 139957634799168 [version_set.cc:6251] Recovered from manifest file:/worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34954/att-00/MANIFEST-206347 succeeded,manifest_file_number is 206347, next_file_number is 278844, last_sequence is 4238025445, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 278714 2024/11/23-20:14:00.374585 139957634799168 [version_set.cc:6266] Column family [default] (ID 0), log number is 0 2024/11/23-20:14:00.374829 139957634799168 [version_set.cc:5721] Creating manifest 278844 2024/11/23-20:14:00.410432 139957634799168 DEBG dcompact_worker.cpp:1218: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00: bottommost_level: fake = 1, rpc = 0 2024/11/23-20:14:00.434829 139957634799168 [compaction_job.cc:2437] [default] [JOB 34954] Compacting 1@2 + 22@3 files to L3, score 0.99, subcompactions 1 : 1 2024/11/23-20:14:00.434849 139957634799168 [compaction_job.cc:2445] [default]: Compaction start summary: Base version 26165 Base level 2, inputs: [278840(31MB)], [278760(40MB) 278761(40MB) 278762(40MB) 278763(40MB) 278764(40MB) 278765(40MB) 278766(40MB) 278767(40MB) 278768(40MB) 278769(40MB) 278770(40MB) 278771(40MB) 278772(40MB) 278773(40MB) 278774(40MB) 278775(40MB) 278776(40MB) 278777(40MB) 278778(40MB) 278779(40MB) 278780(40MB) 278781(40MB)] 2024/11/23-20:14:00.442017 139957634799168 [top_zip_table.cc:59] core git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:14:00.442042 139957634799168 [top_zip_table.cc:60] fsa git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:14:00.442057 139957634799168 [top_zip_table.cc:61] zbs git_version_hash_info_is:commit fc4ab4f2fbfdcdfa23a955d69e26d2fc533b902d Author: leipeng Date: 2024-04-16 20:29:43 +0800 test_sso.cpp: test swap g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. INCS = -Isrc -I3rdparty/re2 -I3rdparty/zstd -Iboost-include CXXFLAGS = -D TOPLING_IO_WITH_URING=1 -fPIC -time -Wno-class-memaccess -fdiagnostics-color -Wformat=2 -Wcomment -Wall -Wextra -Wno-unused-parameter -Wno-alloc-size-larger-than -mcx16 -Wno-deprecated-declarations -Wstrict-aliasing=3 -DCLIENT_LICENSE_ID="public" -DNO_THREADS -ftls-model=initial-exec -DDIVSUFSORT_API= -fno-stack-protector RLS_FLAGS = -O3 -DNDEBUG -g3 WITH_BMI2 = 1 compile_cpu_flag: -march=haswell -mbmi -mbmi2 2024/11/23-20:14:00.442063 139957634799168 [top_zip_table.cc:62] topling-zip_table_reader: git_version_hash_info_is: commit a4a382006cb82713acf5b3425835f22711877681 Author: leipeng Date: 2024-04-11 19:44:14 +0800 fixed_len_hole_index.cc: Add env bool FixedLenHoleIndexUseAVX512 for ut & debug 1. FixedLenHoleIndexUseAVX512 is only in unit tests or debug 2. In release mode, use avx512 by compile options(-march=) g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-20:14:00.442073 139957634799168 [top_zip_table.cc:64] topling-rocks git_version_hash_info_is: commit feab587e468621ba0204d47b4ffacc52e7c4290d Author: leipeng Date: 2024-03-22 13:35:06 +0800 ToplingZipTableBuilder::Add: immediately return after error This function returns void, and caller does not check builder->status() after calling this function and graceful abandon. So immediately return after error, and return error status in Finish(). g++ (GCC) 12.1.0 Copyright (C) 2022 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. cpu_flag: -march=native -mbmi -mbmi2 2024/11/23-20:14:01.172830 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278845.sst) /dev/shm/ToplingTemp/Topling-176696-H9KLQ9 2024/11/23-20:14:01.466988 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278845.sst): first pass time = 0.29's, 456.385'MB/sec 2024/11/23-20:14:02.397615 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278845.sst): old prealloc_size = 147639500, real_size = 40585264 2024/11/23-20:14:02.397750 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278845.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.79 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.52 2024/11/23-20:14:02.398139 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278845.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:02.651893 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278845.sst): second pass time = 0.93's, 141.703'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.25's, 166.837'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1966.671'MB/sec (index lex order gen) rebuild zvType time = 0.19's, 0.174'MB/sec write SST data time = 0.07's, 617.645'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 47828.797'MB/sec zip my value time = 0.93's, unzip length = 0.132'GB zip my value throughput = 141.703'MB/sec zip pipeline throughput = 141.703'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.79 avg-val = 1016.99 avg-zval = 311.52 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.79 avg-val = 1023.99 avg-zval = 311.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6873 value = 3.2646 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1495 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 8.981155 GB avg = 0.171 KB (by entry num) total key len = 1.443410 GB avg = 0.028 KB total ukey len = 0.999431 GB avg = 0.015 KB total ukey num = 0.065679865 Billion total entry num = 0.052487594 Billion write speed all = 0.255940409 MB/sec (with seq num) write speed all = 0.245631147 MB/sec (without seq num) 2024/11/23-20:14:02.719762 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278845: 129430 keys, 42417628 bytes, temperature: kWarm 2024/11/23-20:14:02.719893 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364042719811, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278845, "file_size": 42417628, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237877277, "largest_seqno": 4238004893, "table_properties": {"data_size": 40320480, "index_size": 232256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629264, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364041, "oldest_key_time": 0, "file_creation_time": 1732364041, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278845, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:02.803944 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278846.sst) /dev/shm/ToplingTemp/Topling-176696-Bh5ip9 2024/11/23-20:14:03.452328 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278846.sst): first pass time = 0.65's, 207.024'MB/sec 2024/11/23-20:14:04.415552 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278846.sst): old prealloc_size = 147639500, real_size = 40868416 2024/11/23-20:14:04.415674 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278846.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.97 avg-zval = 313.77 2024/11/23-20:14:04.415825 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278846.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:04.585487 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278846.sst): second pass time = 0.96's, 136.865'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 251.296'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2291.528'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.262'MB/sec write SST data time = 0.05's, 918.574'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48059.082'MB/sec zip my value time = 0.96's, unzip length = 0.132'GB zip my value throughput = 136.865'MB/sec zip pipeline throughput = 136.865'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1016.97 avg-zval = 313.77 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1023.97 avg-zval = 313.77 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9240 value = 3.2412 dict = 1.00 all = 3.1432 } Zip/UnZip{ index = 0.1444 value = 0.3085 dict = 1.00 all = 0.3181 } ---------------------------- total value len = 9.113690 GB avg = 0.173 KB (by entry num) total key len = 1.445998 GB avg = 0.027 KB total ukey len = 1.000984 GB avg = 0.015 KB total ukey num = 0.065809298 Billion total entry num = 0.052617027 Billion write speed all = 0.259245617 MB/sec (with seq num) write speed all = 0.248911423 MB/sec (without seq num) 2024/11/23-20:14:04.639429 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278846: 129433 keys, 42701149 bytes, temperature: kWarm 2024/11/23-20:14:04.639492 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364044639459, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278846, "file_size": 42701149, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868195, "largest_seqno": 4238003385, "table_properties": {"data_size": 40611568, "index_size": 224320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629183, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364042, "oldest_key_time": 0, "file_creation_time": 1732364042, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278846, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:04.678155 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278847.sst) /dev/shm/ToplingTemp/Topling-176696-59ic77 2024/11/23-20:14:05.289630 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278847.sst): first pass time = 0.61's, 219.509'MB/sec 2024/11/23-20:14:06.241104 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278847.sst): old prealloc_size = 147639500, real_size = 40588864 2024/11/23-20:14:06.241277 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278847.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129433 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.79 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.97 avg-zval = 311.54 2024/11/23-20:14:06.241481 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278847.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:06.432444 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278847.sst): second pass time = 0.95's, 138.548'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.19's, 221.714'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1603.037'MB/sec (index lex order gen) rebuild zvType time = 0.14's, 0.229'MB/sec write SST data time = 0.05's, 847.740'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45390.777'MB/sec zip my value time = 0.95's, unzip length = 0.132'GB zip my value throughput = 138.548'MB/sec zip pipeline throughput = 159.607'MB/sec entries = 129433 avg-key = 20.00 avg-zkey = 1.79 avg-val = 1016.97 avg-zval = 311.54 usrkeys = 129433 avg-key = 12.00 avg-zkey = 1.79 avg-val = 1023.97 avg-zval = 311.54 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129433 bytes = 485392 } seq expand size = 906031 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6874 value = 3.2643 dict = 1.00 all = 3.1639 } Zip/UnZip{ index = 0.1495 value = 0.3063 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 9.246226 GB avg = 0.175 KB (by entry num) total key len = 1.448587 GB avg = 0.027 KB total ukey len = 1.002538 GB avg = 0.015 KB total ukey num = 0.065938731 Billion total entry num = 0.052746460 Billion write speed all = 0.262551078 MB/sec (with seq num) write speed all = 0.252191933 MB/sec (without seq num) 2024/11/23-20:14:06.485574 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278847: 129433 keys, 42421711 bytes, temperature: kWarm 2024/11/23-20:14:06.485636 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364046485602, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278847, "file_size": 42421711, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874654, "largest_seqno": 4238004822, "table_properties": {"data_size": 40324080, "index_size": 232256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588660, "raw_average_key_size": 20, "raw_value_size": 131629494, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129433, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364044, "oldest_key_time": 0, "file_creation_time": 1732364044, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278847, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:06.527778 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278848.sst) /dev/shm/ToplingTemp/Topling-176696-yMYHP6 2024/11/23-20:14:07.115732 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278848.sst): first pass time = 0.59's, 228.296'MB/sec 2024/11/23-20:14:09.289272 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278848.sst): old prealloc_size = 147639500, real_size = 40853008 2024/11/23-20:14:09.289389 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278848.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129436 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.74 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.95 avg-zval = 313.64 2024/11/23-20:14:09.289560 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278848.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:09.457471 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278848.sst): second pass time = 2.17's, 60.580'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 253.780'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2262.795'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.268'MB/sec write SST data time = 0.05's, 905.624'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 33524.989'MB/sec zip my value time = 2.17's, unzip length = 0.132'GB zip my value throughput = 60.580'MB/sec zip pipeline throughput = 120.524'MB/sec entries = 129436 avg-key = 20.00 avg-zkey = 1.74 avg-val = 1016.95 avg-zval = 313.64 usrkeys = 129436 avg-key = 12.00 avg-zkey = 1.74 avg-val = 1023.95 avg-zval = 313.64 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129436 bytes = 485392 } seq expand size = 906052 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9104 value = 3.2424 dict = 1.00 all = 3.1443 } Zip/UnZip{ index = 0.1447 value = 0.3084 dict = 1.00 all = 0.3180 } ---------------------------- total value len = 9.378761 GB avg = 0.177 KB (by entry num) total key len = 1.451176 GB avg = 0.027 KB total ukey len = 1.004091 GB avg = 0.015 KB total ukey num = 0.066068167 Billion total entry num = 0.052875896 Billion write speed all = 0.265848555 MB/sec (with seq num) write speed all = 0.255464760 MB/sec (without seq num) 2024/11/23-20:14:09.510507 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278848: 129436 keys, 42685585 bytes, temperature: kWarm 2024/11/23-20:14:09.510574 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364049510542, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278848, "file_size": 42685585, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868596, "largest_seqno": 4237999672, "table_properties": {"data_size": 40595712, "index_size": 224768, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588720, "raw_average_key_size": 20, "raw_value_size": 131629554, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129436, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364046, "oldest_key_time": 0, "file_creation_time": 1732364046, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278848, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:09.552167 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278849.sst) /dev/shm/ToplingTemp/Topling-176696-LjbzZ9 2024/11/23-20:14:10.223342 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278849.sst): first pass time = 0.67's, 199.987'MB/sec 2024/11/23-20:14:11.115531 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278849.sst): old prealloc_size = 147639500, real_size = 40610720 2024/11/23-20:14:11.115657 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278849.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.79 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.98 avg-zval = 311.72 2024/11/23-20:14:11.115825 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278849.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:11.276325 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278849.sst): second pass time = 0.89's, 147.636'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.998'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2250.442'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.278'MB/sec write SST data time = 0.04's, 959.620'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49936.126'MB/sec zip my value time = 0.89's, unzip length = 0.132'GB zip my value throughput = 147.636'MB/sec zip pipeline throughput = 147.636'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.79 avg-val = 1016.98 avg-zval = 311.72 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.79 avg-val = 1023.98 avg-zval = 311.72 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129432 bytes = 485392 } seq expand size = 906024 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6985 value = 3.2625 dict = 1.00 all = 3.1623 } Zip/UnZip{ index = 0.1493 value = 0.3065 dict = 1.00 all = 0.3162 } ---------------------------- total value len = 9.511297 GB avg = 0.179 KB (by entry num) total key len = 1.453764 GB avg = 0.027 KB total ukey len = 1.005644 GB avg = 0.015 KB total ukey num = 0.066197599 Billion total entry num = 0.053005328 Billion write speed all = 0.269153504 MB/sec (with seq num) write speed all = 0.258744756 MB/sec (without seq num) 2024/11/23-20:14:11.328339 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278849: 129432 keys, 42443758 bytes, temperature: kWarm 2024/11/23-20:14:11.328400 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364051328367, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278849, "file_size": 42443758, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237869923, "largest_seqno": 4238001502, "table_properties": {"data_size": 40346320, "index_size": 231872, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629397, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364049, "oldest_key_time": 0, "file_creation_time": 1732364049, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278849, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:11.365865 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278850.sst) /dev/shm/ToplingTemp/Topling-176696-7nEsT8 2024/11/23-20:14:11.931861 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278850.sst): first pass time = 0.57's, 237.152'MB/sec 2024/11/23-20:14:14.002570 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278850.sst): old prealloc_size = 147639500, real_size = 40584320 2024/11/23-20:14:14.002698 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278850.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.79 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.52 2024/11/23-20:14:14.002877 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278850.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:14.168493 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278850.sst): second pass time = 2.07's, 63.588'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 255.657'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2195.717'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.272'MB/sec write SST data time = 0.05's, 907.469'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 49920.917'MB/sec zip my value time = 2.07's, unzip length = 0.132'GB zip my value throughput = 63.588'MB/sec zip pipeline throughput = 113.985'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.79 avg-val = 1016.99 avg-zval = 311.52 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.79 avg-val = 1023.99 avg-zval = 311.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 14 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6854 value = 3.2647 dict = 1.00 all = 3.1642 } Zip/UnZip{ index = 0.1496 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 9.643832 GB avg = 0.181 KB (by entry num) total key len = 1.456353 GB avg = 0.027 KB total ukey len = 1.007197 GB avg = 0.015 KB total ukey num = 0.066327029 Billion total entry num = 0.053134758 Billion write speed all = 0.272450977 MB/sec (with seq num) write speed all = 0.262017553 MB/sec (without seq num) 2024/11/23-20:14:14.264141 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278850: 129430 keys, 42417120 bytes, temperature: kWarm 2024/11/23-20:14:14.264208 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364054264174, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278850, "file_size": 42417120, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872175, "largest_seqno": 4237993094, "table_properties": {"data_size": 40319472, "index_size": 232320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629313, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364051, "oldest_key_time": 0, "file_creation_time": 1732364051, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278850, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:14.305428 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278851.sst) /dev/shm/ToplingTemp/Topling-176696-G0YMa9 2024/11/23-20:14:14.917483 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278851.sst): first pass time = 0.61's, 219.308'MB/sec 2024/11/23-20:14:15.766400 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278851.sst): old prealloc_size = 147639500, real_size = 40801616 2024/11/23-20:14:15.766526 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278851.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.01 avg-zval = 313.23 2024/11/23-20:14:15.766685 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278851.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:15.932728 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278851.sst): second pass time = 0.85's, 155.376'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 256.360'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2262.236'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 894.020'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43553.814'MB/sec zip my value time = 0.85's, unzip length = 0.132'GB zip my value throughput = 155.376'MB/sec zip pipeline throughput = 155.376'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1017.01 avg-zval = 313.23 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1024.01 avg-zval = 313.23 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129429 bytes = 485376 } seq expand size = 906003 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7939 value = 3.2469 dict = 1.00 all = 3.1481 } Zip/UnZip{ index = 0.1472 value = 0.3080 dict = 1.00 all = 0.3177 } ---------------------------- total value len = 9.776368 GB avg = 0.184 KB (by entry num) total key len = 1.458942 GB avg = 0.027 KB total ukey len = 1.008750 GB avg = 0.015 KB total ukey num = 0.066456458 Billion total entry num = 0.053264187 Billion write speed all = 0.275755635 MB/sec (with seq num) write speed all = 0.265297249 MB/sec (without seq num) 2024/11/23-20:14:15.987216 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278851: 129429 keys, 42634687 bytes, temperature: kWarm 2024/11/23-20:14:15.987293 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364055987260, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278851, "file_size": 42634687, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237885133, "largest_seqno": 4238004913, "table_properties": {"data_size": 40540480, "index_size": 228608, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630133, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364054, "oldest_key_time": 0, "file_creation_time": 1732364054, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278851, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:16.026480 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278852.sst) /dev/shm/ToplingTemp/Topling-176696-p06MJ5 2024/11/23-20:14:16.928649 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278852.sst): first pass time = 0.90's, 148.779'MB/sec 2024/11/23-20:14:18.002731 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278852.sst): old prealloc_size = 147639500, real_size = 40994944 2024/11/23-20:14:18.002855 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278852.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1017.01 avg-zval = 314.76 2024/11/23-20:14:18.003036 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278852.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:18.167507 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278852.sst): second pass time = 1.07's, 122.657'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.957'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2307.632'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.05's, 916.933'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 23782.756'MB/sec zip my value time = 1.07's, unzip length = 0.132'GB zip my value throughput = 122.657'MB/sec zip pipeline throughput = 132.975'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.01 avg-zval = 314.76 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1024.01 avg-zval = 314.76 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129428 bytes = 485376 } seq expand size = 905996 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9396 value = 3.2311 dict = 1.00 all = 3.1339 } Zip/UnZip{ index = 0.1441 value = 0.3095 dict = 1.00 all = 0.3191 } ---------------------------- total value len = 9.908904 GB avg = 0.186 KB (by entry num) total key len = 1.461530 GB avg = 0.027 KB total ukey len = 1.010303 GB avg = 0.015 KB total ukey num = 0.066585886 Billion total entry num = 0.053393615 Billion write speed all = 0.279056778 MB/sec (with seq num) write speed all = 0.268573555 MB/sec (without seq num) 2024/11/23-20:14:18.221823 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278852: 129428 keys, 42827795 bytes, temperature: kWarm 2024/11/23-20:14:18.221889 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364058221857, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278852, "file_size": 42827795, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237880987, "largest_seqno": 4238003551, "table_properties": {"data_size": 40738608, "index_size": 223808, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629948, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364056, "oldest_key_time": 0, "file_creation_time": 1732364056, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278852, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:18.260903 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278853.sst) /dev/shm/ToplingTemp/Topling-176696-JOF6k8 2024/11/23-20:14:19.013298 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278853.sst): first pass time = 0.75's, 178.396'MB/sec 2024/11/23-20:14:19.853352 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278853.sst): old prealloc_size = 147639500, real_size = 41020736 2024/11/23-20:14:19.853471 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278853.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.71 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.99 avg-zval = 314.98 2024/11/23-20:14:19.853634 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278853.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:20.016217 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278853.sst): second pass time = 0.84's, 156.831'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.152'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2317.770'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.274'MB/sec write SST data time = 0.04's, 961.637'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39910.673'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 156.831'MB/sec zip pipeline throughput = 156.831'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.71 avg-val = 1016.99 avg-zval = 314.98 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.71 avg-val = 1023.99 avg-zval = 314.98 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 7.0342 value = 3.2288 dict = 1.00 all = 3.1320 } Zip/UnZip{ index = 0.1422 value = 0.3097 dict = 1.00 all = 0.3193 } ---------------------------- total value len = 10.041439 GB avg = 0.188 KB (by entry num) total key len = 1.464119 GB avg = 0.027 KB total ukey len = 1.011857 GB avg = 0.015 KB total ukey num = 0.066715316 Billion total entry num = 0.053523045 Billion write speed all = 0.282360217 MB/sec (with seq num) write speed all = 0.271852059 MB/sec (without seq num) 2024/11/23-20:14:20.068153 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278853: 129430 keys, 42853435 bytes, temperature: kWarm 2024/11/23-20:14:20.068224 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364060068188, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278853, "file_size": 42853435, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237868290, "largest_seqno": 4238005006, "table_properties": {"data_size": 40767408, "index_size": 220800, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629163, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364058, "oldest_key_time": 0, "file_creation_time": 1732364058, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278853, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:20.107158 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278854.sst) /dev/shm/ToplingTemp/Topling-176696-4FbaW8 2024/11/23-20:14:20.837693 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278854.sst): first pass time = 0.73's, 183.735'MB/sec 2024/11/23-20:14:21.683159 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278854.sst): old prealloc_size = 147639500, real_size = 40819760 2024/11/23-20:14:21.683301 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278854.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.76 raw-val = 0.1316 GB zip-val = 0.0406 GB avg-val =1016.99 avg-zval = 313.37 2024/11/23-20:14:21.683464 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278854.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:21.848902 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278854.sst): second pass time = 0.84's, 156.056'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 257.382'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1913.344'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.05's, 907.559'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44015.875'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 156.056'MB/sec zip pipeline throughput = 156.056'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.76 avg-val = 1016.99 avg-zval = 313.37 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.76 avg-val = 1023.99 avg-zval = 313.37 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129431 bytes = 485376 } seq expand size = 906017 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0406 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.8189 value = 3.2454 dict = 1.00 all = 3.1468 } Zip/UnZip{ index = 0.1467 value = 0.3081 dict = 1.00 all = 0.3178 } ---------------------------- total value len = 10.173975 GB avg = 0.190 KB (by entry num) total key len = 1.466707 GB avg = 0.027 KB total ukey len = 1.013410 GB avg = 0.015 KB total ukey num = 0.066844747 Billion total entry num = 0.053652476 Billion write speed all = 0.285663482 MB/sec (with seq num) write speed all = 0.275130386 MB/sec (without seq num) 2024/11/23-20:14:21.902815 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278854: 129431 keys, 42652608 bytes, temperature: kWarm 2024/11/23-20:14:21.902898 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364061902849, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278854, "file_size": 42652608, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237874127, "largest_seqno": 4238005012, "table_properties": {"data_size": 40559456, "index_size": 227776, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629667, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364060, "oldest_key_time": 0, "file_creation_time": 1732364060, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278854, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:21.941293 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278855.sst) /dev/shm/ToplingTemp/Topling-176696-ryfkD7 2024/11/23-20:14:22.755369 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278855.sst): first pass time = 0.81's, 164.880'MB/sec 2024/11/23-20:14:23.569760 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278855.sst): old prealloc_size = 147639500, real_size = 40590288 2024/11/23-20:14:23.569885 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278855.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129432 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.79 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.98 avg-zval = 311.56 2024/11/23-20:14:23.570060 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278855.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:23.731252 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278855.sst): second pass time = 0.81's, 161.754'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 262.737'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2227.533'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.277'MB/sec write SST data time = 0.04's, 948.359'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45133.014'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 161.754'MB/sec zip pipeline throughput = 161.754'MB/sec entries = 129432 avg-key = 20.00 avg-zkey = 1.79 avg-val = 1016.98 avg-zval = 311.56 usrkeys = 129432 avg-key = 12.00 avg-zkey = 1.79 avg-val = 1023.98 avg-zval = 311.56 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129432 bytes = 485392 } seq expand size = 906024 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6874 value = 3.2642 dict = 1.00 all = 3.1638 } Zip/UnZip{ index = 0.1495 value = 0.3064 dict = 1.00 all = 0.3161 } ---------------------------- total value len = 10.306511 GB avg = 0.192 KB (by entry num) total key len = 1.469296 GB avg = 0.027 KB total ukey len = 1.014963 GB avg = 0.015 KB total ukey num = 0.066974179 Billion total entry num = 0.053781908 Billion write speed all = 0.288966106 MB/sec (with seq num) write speed all = 0.278408087 MB/sec (without seq num) 2024/11/23-20:14:23.783771 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278855: 129432 keys, 42422650 bytes, temperature: kWarm 2024/11/23-20:14:23.783839 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364063783805, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278855, "file_size": 42422650, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237873877, "largest_seqno": 4238005183, "table_properties": {"data_size": 40325504, "index_size": 232256, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588640, "raw_average_key_size": 20, "raw_value_size": 131629984, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129432, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364061, "oldest_key_time": 0, "file_creation_time": 1732364061, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278855, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:23.820202 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278856.sst) /dev/shm/ToplingTemp/Topling-176696-glUbp9 2024/11/23-20:14:24.412362 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278856.sst): first pass time = 0.59's, 226.677'MB/sec 2024/11/23-20:14:25.232867 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278856.sst): old prealloc_size = 147639500, real_size = 40583616 2024/11/23-20:14:25.233011 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278856.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129428 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.79 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.01 avg-zval = 311.52 2024/11/23-20:14:25.233172 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278856.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:25.395073 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278856.sst): second pass time = 0.81's, 162.176'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.520'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1893.198'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 951.992'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 43260.893'MB/sec zip my value time = 0.81's, unzip length = 0.132'GB zip my value throughput = 162.176'MB/sec zip pipeline throughput = 162.176'MB/sec entries = 129428 avg-key = 20.00 avg-zkey = 1.79 avg-val = 1017.01 avg-zval = 311.52 usrkeys = 129428 avg-key = 12.00 avg-zkey = 1.79 avg-val = 1024.01 avg-zval = 311.52 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129428 bytes = 485376 } seq expand size = 905996 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0424 GB } UnZip/Zip{ index = 6.6853 value = 3.2647 dict = 1.00 all = 3.1643 } Zip/UnZip{ index = 0.1496 value = 0.3063 dict = 1.00 all = 0.3160 } ---------------------------- total value len = 10.439046 GB avg = 0.194 KB (by entry num) total key len = 1.471885 GB avg = 0.027 KB total ukey len = 1.016516 GB avg = 0.015 KB total ukey num = 0.067103607 Billion total entry num = 0.053911336 Billion write speed all = 0.292269969 MB/sec (with seq num) write speed all = 0.281686975 MB/sec (without seq num) 2024/11/23-20:14:25.447845 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278856: 129428 keys, 42416518 bytes, temperature: kWarm 2024/11/23-20:14:25.447916 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364065447881, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278856, "file_size": 42416518, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237872082, "largest_seqno": 4238001503, "table_properties": {"data_size": 40318768, "index_size": 232320, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588560, "raw_average_key_size": 20, "raw_value_size": 131629184, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129428, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364063, "oldest_key_time": 0, "file_creation_time": 1732364063, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278856, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:25.484820 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278857.sst) /dev/shm/ToplingTemp/Topling-176696-LQoTE8 2024/11/23-20:14:26.030578 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278857.sst): first pass time = 0.55's, 245.948'MB/sec 2024/11/23-20:14:27.060147 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278857.sst): old prealloc_size = 147639500, real_size = 41006864 2024/11/23-20:14:27.060307 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278857.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1016.99 avg-zval = 314.84 2024/11/23-20:14:27.060466 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278857.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:27.220467 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278857.sst): second pass time = 1.03's, 127.938'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 267.249'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2051.988'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 999.451'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36640.596'MB/sec zip my value time = 1.03's, unzip length = 0.132'GB zip my value throughput = 127.938'MB/sec zip pipeline throughput = 195.879'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1016.99 avg-zval = 314.84 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1023.99 avg-zval = 314.84 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129430 bytes = 485376 } seq expand size = 906010 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0428 GB } UnZip/Zip{ index = 6.9278 value = 3.2302 dict = 1.00 all = 3.1330 } Zip/UnZip{ index = 0.1443 value = 0.3096 dict = 1.00 all = 0.3192 } ---------------------------- total value len = 10.571582 GB avg = 0.196 KB (by entry num) total key len = 1.474473 GB avg = 0.027 KB total ukey len = 1.018069 GB avg = 0.015 KB total ukey num = 0.067233037 Billion total entry num = 0.054040766 Billion write speed all = 0.295572404 MB/sec (with seq num) write speed all = 0.284964477 MB/sec (without seq num) 2024/11/23-20:14:27.271460 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278857: 129430 keys, 42839867 bytes, temperature: kWarm 2024/11/23-20:14:27.271527 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364067271492, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278857, "file_size": 42839867, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237888398, "largest_seqno": 4237996003, "table_properties": {"data_size": 40750144, "index_size": 224192, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629620, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364065, "oldest_key_time": 0, "file_creation_time": 1732364065, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278857, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:27.310802 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278858.sst) /dev/shm/ToplingTemp/Topling-176696-smXAG8 2024/11/23-20:14:27.878913 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278858.sst): first pass time = 0.57's, 236.268'MB/sec 2024/11/23-20:14:28.651743 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278858.sst): old prealloc_size = 147639500, real_size = 40718464 2024/11/23-20:14:28.651859 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278858.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129424 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.78 raw-val = 0.1316 GB zip-val = 0.0405 GB avg-val =1017.04 avg-zval = 312.58 2024/11/23-20:14:28.652031 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278858.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:28.814779 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278858.sst): second pass time = 0.77's, 170.479'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 261.033'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2404.647'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.269'MB/sec write SST data time = 0.04's, 997.896'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44161.802'MB/sec zip my value time = 0.77's, unzip length = 0.132'GB zip my value throughput = 170.479'MB/sec zip pipeline throughput = 170.479'MB/sec entries = 129424 avg-key = 20.00 avg-zkey = 1.78 avg-val = 1017.04 avg-zval = 312.58 usrkeys = 129424 avg-key = 12.00 avg-zkey = 1.78 avg-val = 1024.04 avg-zval = 312.58 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 129424 bytes = 485360 } seq expand size = 905968 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0405 GB dict = 1.38 MB all = 0.0426 GB } UnZip/Zip{ index = 6.7483 value = 3.2537 dict = 1.00 all = 3.1542 } Zip/UnZip{ index = 0.1482 value = 0.3073 dict = 1.00 all = 0.3170 } ---------------------------- total value len = 10.704117 GB avg = 0.198 KB (by entry num) total key len = 1.477062 GB avg = 0.027 KB total ukey len = 1.019622 GB avg = 0.015 KB total ukey num = 0.067362461 Billion total entry num = 0.054170190 Billion write speed all = 0.298876229 MB/sec (with seq num) write speed all = 0.288243312 MB/sec (without seq num) 2024/11/23-20:14:28.869361 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278858: 129424 keys, 42551517 bytes, temperature: kWarm 2024/11/23-20:14:28.869420 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364068869386, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278858, "file_size": 42551517, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 3237871365, "largest_seqno": 4238002155, "table_properties": {"data_size": 40455792, "index_size": 230144, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588480, "raw_average_key_size": 20, "raw_value_size": 131629468, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129424, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364067, "oldest_key_time": 0, "file_creation_time": 1732364067, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278858, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:28.906340 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278859.sst) /dev/shm/ToplingTemp/Topling-176696-9qOAm5 2024/11/23-20:14:29.438567 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278859.sst): first pass time = 0.53's, 252.197'MB/sec 2024/11/23-20:14:30.224038 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278859.sst): old prealloc_size = 147639500, real_size = 40542784 2024/11/23-20:14:30.224162 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278859.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.75 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1016.99 avg-zval = 311.24 2024/11/23-20:14:30.224298 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278859.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:30.405989 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278859.sst): second pass time = 0.78's, 167.731'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.18's, 231.939'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2279.470'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.06's, 665.486'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 44322.329'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 167.731'MB/sec zip pipeline throughput = 167.731'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.75 avg-val = 1016.99 avg-zval = 311.24 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.75 avg-val = 1021.02 avg-zval = 311.24 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 74552 bytes = 279584 } seq expand size = 521864 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 6.8690 value = 3.2675 dict = 1.00 all = 3.1813 } Zip/UnZip{ index = 0.1456 value = 0.3060 dict = 1.00 all = 0.3143 } ---------------------------- total value len = 10.836268 GB avg = 0.200 KB (by entry num) total key len = 1.479650 GB avg = 0.027 KB total ukey len = 1.021175 GB avg = 0.015 KB total ukey num = 0.067491891 Billion total entry num = 0.054299620 Billion write speed all = 0.302170390 MB/sec (with seq num) write speed all = 0.291512484 MB/sec (without seq num) 2024/11/23-20:14:30.462371 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278859: 129430 keys, 42189633 bytes, temperature: kWarm 2024/11/23-20:14:30.462460 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364070462413, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278859, "file_size": 42189633, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4237989668, "table_properties": {"data_size": 40284144, "index_size": 226112, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629137, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364068, "oldest_key_time": 0, "file_creation_time": 1732364068, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278859, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:30.514934 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278860.sst) /dev/shm/ToplingTemp/Topling-176696-TRyMq5 2024/11/23-20:14:31.127501 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278860.sst): first pass time = 0.61's, 219.132'MB/sec 2024/11/23-20:14:31.923579 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278860.sst): old prealloc_size = 147639500, real_size = 40607936 2024/11/23-20:14:31.923698 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278860.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.73 raw-val = 0.1316 GB zip-val = 0.0404 GB avg-val =1017.02 avg-zval = 311.77 2024/11/23-20:14:31.923818 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278860.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:32.089491 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278860.sst): second pass time = 0.79's, 165.681'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.17's, 254.382'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2276.420'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.270'MB/sec write SST data time = 0.05's, 914.768'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 39503.584'MB/sec zip my value time = 0.79's, unzip length = 0.132'GB zip my value throughput = 165.681'MB/sec zip pipeline throughput = 165.681'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.73 avg-val = 1017.02 avg-zval = 311.77 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.73 avg-val = 1020.30 avg-zval = 311.77 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 60576 bytes = 227168 } seq expand size = 424032 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0404 GB dict = 1.38 MB all = 0.0422 GB } UnZip/Zip{ index = 6.9356 value = 3.2621 dict = 1.00 all = 3.1803 } Zip/UnZip{ index = 0.1442 value = 0.3066 dict = 1.00 all = 0.3144 } ---------------------------- total value len = 10.968323 GB avg = 0.202 KB (by entry num) total key len = 1.482239 GB avg = 0.027 KB total ukey len = 1.022729 GB avg = 0.015 KB total ukey num = 0.067621318 Billion total entry num = 0.054429047 Billion write speed all = 0.305461224 MB/sec (with seq num) write speed all = 0.294778356 MB/sec (without seq num) 2024/11/23-20:14:32.175405 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278860: 129427 keys, 42202795 bytes, temperature: kWarm 2024/11/23-20:14:32.175451 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364072175424, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278860, "file_size": 42202795, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4238003386, "table_properties": {"data_size": 40351472, "index_size": 223936, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131630147, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364070, "oldest_key_time": 0, "file_creation_time": 1732364070, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278860, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:32.220924 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278861.sst) /dev/shm/ToplingTemp/Topling-176696-fQJfr8 2024/11/23-20:14:32.797152 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278861.sst): first pass time = 0.58's, 232.939'MB/sec 2024/11/23-20:14:33.642466 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278861.sst): old prealloc_size = 147639500, real_size = 41077136 2024/11/23-20:14:33.642591 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278861.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129426 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0408 GB avg-val =1017.03 avg-zval = 315.41 2024/11/23-20:14:33.642756 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278861.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:33.805044 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278861.sst): second pass time = 0.84's, 155.949'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.973'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2192.170'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 962.487'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 46617.232'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 155.949'MB/sec zip pipeline throughput = 155.949'MB/sec entries = 129426 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.03 avg-zval = 315.41 usrkeys = 129426 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1023.75 avg-zval = 315.41 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 124188 bytes = 465712 } seq expand size = 869316 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0408 GB dict = 1.38 MB all = 0.0429 GB } UnZip/Zip{ index = 6.9935 value = 3.2244 dict = 1.00 all = 3.1279 } Zip/UnZip{ index = 0.1430 value = 0.3101 dict = 1.00 all = 0.3197 } ---------------------------- total value len = 11.100822 GB avg = 0.203 KB (by entry num) total key len = 1.484827 GB avg = 0.027 KB total ukey len = 1.024282 GB avg = 0.015 KB total ukey num = 0.067750744 Billion total entry num = 0.054558473 Billion write speed all = 0.308762464 MB/sec (with seq num) write speed all = 0.298054643 MB/sec (without seq num) 2024/11/23-20:14:33.856683 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278861: 129426 keys, 42910595 bytes, temperature: kWarm 2024/11/23-20:14:33.856778 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364073856727, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278861, "file_size": 42910595, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4237984142, "table_properties": {"data_size": 40822528, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588520, "raw_average_key_size": 20, "raw_value_size": 131629967, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129426, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364072, "oldest_key_time": 0, "file_creation_time": 1732364072, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278861, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:33.911316 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278862.sst) /dev/shm/ToplingTemp/Topling-176696-wp7GA8 2024/11/23-20:14:34.655178 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278862.sst): first pass time = 0.74's, 180.449'MB/sec 2024/11/23-20:14:35.440770 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278862.sst): old prealloc_size = 147639500, real_size = 40566208 2024/11/23-20:14:35.440895 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278862.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.77 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.00 avg-zval = 311.40 2024/11/23-20:14:35.441045 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278862.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:35.601357 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278862.sst): second pass time = 0.78's, 167.701'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.343'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2196.181'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 986.623'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.379'MB, throughput = 35250.524'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 167.701'MB/sec zip pipeline throughput = 167.701'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.77 avg-val = 1017.00 avg-zval = 311.40 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.77 avg-val = 1022.03 avg-zval = 311.40 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 93045 bytes = 348928 } seq expand size = 651315 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0423 GB } UnZip/Zip{ index = 6.7864 value = 3.2659 dict = 1.00 all = 3.1743 } Zip/UnZip{ index = 0.1474 value = 0.3062 dict = 1.00 all = 0.3150 } ---------------------------- total value len = 11.233103 GB avg = 0.205 KB (by entry num) total key len = 1.487416 GB avg = 0.027 KB total ukey len = 1.025835 GB avg = 0.015 KB total ukey num = 0.067880174 Billion total entry num = 0.054687903 Billion write speed all = 0.312057464 MB/sec (with seq num) write speed all = 0.301324714 MB/sec (without seq num) 2024/11/23-20:14:35.652073 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278862: 129430 keys, 42283466 bytes, temperature: kWarm 2024/11/23-20:14:35.652141 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364075652107, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278862, "file_size": 42283466, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4238004945, "table_properties": {"data_size": 40304816, "index_size": 228864, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131630084, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364073, "oldest_key_time": 0, "file_creation_time": 1732364073, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278862, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:35.691826 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278863.sst) /dev/shm/ToplingTemp/Topling-176696-hU6Fa8 2024/11/23-20:14:36.243140 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278863.sst): first pass time = 0.55's, 243.473'MB/sec 2024/11/23-20:14:37.073899 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278863.sst): old prealloc_size = 147639500, real_size = 40484448 2024/11/23-20:14:37.074036 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278863.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129429 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1017.01 avg-zval = 310.83 2024/11/23-20:14:37.074124 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278863.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:37.233374 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278863.sst): second pass time = 0.83's, 158.722'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 263.285'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1894.347'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.276'MB/sec write SST data time = 0.04's, 995.233'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 42413.861'MB/sec zip my value time = 0.83's, unzip length = 0.132'GB zip my value throughput = 158.722'MB/sec zip pipeline throughput = 158.722'MB/sec entries = 129429 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.01 avg-zval = 310.83 usrkeys = 129429 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.87 avg-zval = 310.83 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 34401 bytes = 129024 } seq expand size = 240807 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9936 value = 3.2720 dict = 1.00 all = 3.1971 } Zip/UnZip{ index = 0.1430 value = 0.3056 dict = 1.00 all = 0.3128 } ---------------------------- total value len = 11.364974 GB avg = 0.207 KB (by entry num) total key len = 1.490004 GB avg = 0.027 KB total ukey len = 1.027388 GB avg = 0.015 KB total ukey num = 0.068009603 Billion total entry num = 0.054817332 Billion write speed all = 0.315343375 MB/sec (with seq num) write speed all = 0.304585655 MB/sec (without seq num) 2024/11/23-20:14:37.290268 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278863: 129429 keys, 41981331 bytes, temperature: kWarm 2024/11/23-20:14:37.290337 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364077290303, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278863, "file_size": 41981331, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4237958694, "table_properties": {"data_size": 40229840, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588580, "raw_average_key_size": 20, "raw_value_size": 131630122, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129429, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364075, "oldest_key_time": 0, "file_creation_time": 1732364075, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278863, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:37.328360 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278864.sst) /dev/shm/ToplingTemp/Topling-176696-zbF888 2024/11/23-20:14:38.044596 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278864.sst): first pass time = 0.72's, 187.405'MB/sec 2024/11/23-20:14:38.825559 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278864.sst): old prealloc_size = 147639500, real_size = 40602656 2024/11/23-20:14:38.825680 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278864.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129427 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0403 GB avg-val =1017.02 avg-zval = 311.74 2024/11/23-20:14:38.825765 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278864.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:38.988034 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278864.sst): second pass time = 0.78's, 168.869'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 259.155'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2237.537'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.267'MB/sec write SST data time = 0.04's, 1022.119'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 48319.438'MB/sec zip my value time = 0.78's, unzip length = 0.132'GB zip my value throughput = 168.869'MB/sec zip pipeline throughput = 168.869'MB/sec entries = 129427 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1017.02 avg-zval = 311.74 usrkeys = 129427 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.86 avg-zval = 311.74 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 34012 bytes = 127552 } seq expand size = 238084 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0403 GB dict = 1.38 MB all = 0.0421 GB } UnZip/Zip{ index = 6.9935 value = 3.2624 dict = 1.00 all = 3.1882 } Zip/UnZip{ index = 0.1430 value = 0.3065 dict = 1.00 all = 0.3137 } ---------------------------- total value len = 11.496842 GB avg = 0.209 KB (by entry num) total key len = 1.492593 GB avg = 0.027 KB total ukey len = 1.028941 GB avg = 0.015 KB total ukey num = 0.068139030 Billion total entry num = 0.054946759 Billion write speed all = 0.318627993 MB/sec (with seq num) write speed all = 0.307845338 MB/sec (without seq num) 2024/11/23-20:14:39.044123 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278864: 129427 keys, 42098038 bytes, temperature: kWarm 2024/11/23-20:14:39.044189 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364079044154, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278864, "file_size": 42098038, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4237966519, "table_properties": {"data_size": 40348048, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588540, "raw_average_key_size": 20, "raw_value_size": 131629983, "raw_average_value_size": 1017, "num_data_blocks": 1, "num_entries": 129427, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364077, "oldest_key_time": 0, "file_creation_time": 1732364077, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278864, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:39.083586 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278865.sst) /dev/shm/ToplingTemp/Topling-176696-tAZ6e7 2024/11/23-20:14:39.783280 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278865.sst): first pass time = 0.70's, 191.834'MB/sec 2024/11/23-20:14:40.624840 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278865.sst): old prealloc_size = 147639500, real_size = 40472800 2024/11/23-20:14:40.624965 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278865.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129430 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0402 GB avg-val =1016.99 avg-zval = 310.73 2024/11/23-20:14:40.625064 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278865.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:40.787527 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278865.sst): second pass time = 0.84's, 156.542'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 257.967'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 1930.963'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.273'MB/sec write SST data time = 0.04's, 956.354'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 36900.600'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 156.542'MB/sec zip pipeline throughput = 156.542'MB/sec entries = 129430 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.99 avg-zval = 310.73 usrkeys = 129430 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1018.80 avg-zval = 310.73 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 33362 bytes = 125120 } seq expand size = 233534 multi value expand size = 0 cnt WriteAppend = 12 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0402 GB dict = 1.38 MB all = 0.0420 GB } UnZip/Zip{ index = 6.9937 value = 3.2729 dict = 1.00 all = 3.1983 } Zip/UnZip{ index = 0.1430 value = 0.3055 dict = 1.00 all = 0.3127 } ---------------------------- total value len = 11.628705 GB avg = 0.211 KB (by entry num) total key len = 1.495182 GB avg = 0.027 KB total ukey len = 1.030494 GB avg = 0.015 KB total ukey num = 0.068268460 Billion total entry num = 0.055076189 Billion write speed all = 0.321911852 MB/sec (with seq num) write speed all = 0.311104274 MB/sec (without seq num) 2024/11/23-20:14:40.845057 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278865: 129430 keys, 41965230 bytes, temperature: kWarm 2024/11/23-20:14:40.845119 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364080845089, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278865, "file_size": 41965230, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4237996550, "table_properties": {"data_size": 40218192, "index_size": 222080, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588600, "raw_average_key_size": 20, "raw_value_size": 131629470, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129430, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364079, "oldest_key_time": 0, "file_creation_time": 1732364079, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278865, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:40.887104 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278866.sst) /dev/shm/ToplingTemp/Topling-176696-LxFQw6 2024/11/23-20:14:41.704339 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278866.sst): first pass time = 0.82's, 164.242'MB/sec 2024/11/23-20:14:42.549309 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278866.sst): old prealloc_size = 147639500, real_size = 40968208 2024/11/23-20:14:42.549433 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278866.sst): index type = NestLoudsTrieDAWG_Mixed_XL_256_32_FL, store type = DictZipBlobStore usrkeys = 129431 prefix = 4 raw-key = 0.0010 GB zip-key = 0.0002 GB avg-key = 8.00 avg-zkey = 1.72 raw-val = 0.1316 GB zip-val = 0.0407 GB avg-val =1016.98 avg-zval = 314.55 2024/11/23-20:14:42.549591 139957634799168 [top_zip_table_builder.cc:1299] /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278866.sst: sumWaitingMem = 0.000 GB, sumWorkingMem = 0.000 GB, reorder workingMem = 0.0003 GB, waited 0.000 sec, Key+Value = 0.134 GB 2024/11/23-20:14:42.711079 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278866.sst): second pass time = 0.84's, 155.919'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.16's, 264.270'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 2215.576'MB/sec (index lex order gen) rebuild zvType time = 0.12's, 0.275'MB/sec write SST data time = 0.04's, 976.342'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 1.378'MB, throughput = 45050.262'MB/sec zip my value time = 0.84's, unzip length = 0.132'GB zip my value throughput = 155.919'MB/sec zip pipeline throughput = 155.919'MB/sec entries = 129431 avg-key = 20.00 avg-zkey = 1.72 avg-val = 1016.98 avg-zval = 314.55 usrkeys = 129431 avg-key = 12.00 avg-zkey = 1.72 avg-val = 1022.90 avg-zval = 314.55 TagRS{ kinds = RS_Key0_TagN bytes = 20320 } TagArray{ size = 109421 bytes = 410336 } seq expand size = 765947 multi value expand size = 0 cnt WriteAppend = 13 UnZipSize{ index = 0.0016 GB value = 0.1316 GB dict = 1.38 MB all = 0.1342 GB } __ZipSize{ index = 0.0002 GB value = 0.0407 GB dict = 1.38 MB all = 0.0427 GB } UnZip/Zip{ index = 6.9797 value = 3.2331 dict = 1.00 all = 3.1399 } Zip/UnZip{ index = 0.1433 value = 0.3093 dict = 1.00 all = 0.3185 } ---------------------------- total value len = 11.761100 GB avg = 0.213 KB (by entry num) total key len = 1.497770 GB avg = 0.027 KB total ukey len = 1.032047 GB avg = 0.015 KB total ukey num = 0.068397891 Billion total entry num = 0.055205620 Billion write speed all = 0.325207484 MB/sec (with seq num) write speed all = 0.314375020 MB/sec (without seq num) 2024/11/23-20:14:42.764672 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278866: 129431 keys, 42745892 bytes, temperature: kWarm 2024/11/23-20:14:42.764743 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364082764707, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278866, "file_size": 42745892, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 0, "largest_seqno": 4238003436, "table_properties": {"data_size": 40713152, "index_size": 222528, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 2588620, "raw_average_key_size": 20, "raw_value_size": 131629229, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 129431, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_TagN-NestLoudsTrieDAWG_Mixed_XL_256_32_FL-DictZipBlobStore", "creation_time": 1732364080, "oldest_key_time": 0, "file_creation_time": 1732364080, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278866, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:42.803785 139957634799168 [top_zip_table_builder.cc:934] ToplingZipTableBuilder(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278867.sst) /dev/shm/ToplingTemp/Topling-176696-liP2o6 2024/11/23-20:14:42.811795 139957634799168 [top_zip_table_builder.cc:1352] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278867.sst): first pass time = 0.01's, 341.684'MB/sec 2024/11/23-20:14:42.842617 139957634799168 [top_zip_table_builder.cc:2772] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278867.sst): old prealloc_size = 147639500, real_size = 867392 2024/11/23-20:14:42.842655 139957634799168 [top_zip_table_builder.cc:1773] BuildReorderMap(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278867.sst): index type = UintIndex_IL_256_32 , store type = DictZipBlobStore usrkeys = 2629 prefix = 4 raw-key = 0.0000 GB zip-key = 0.0000 GB avg-key = 8.00 avg-zkey = 0.30 raw-val = 0.0027 GB zip-val = 0.0009 GB avg-val =1016.87 avg-zval = 329.31 2024/11/23-20:14:42.845703 139957634799168 [top_zip_table_builder.cc:3100] ToplingZipTableBuilder::Finish(/storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00/278867.sst): second pass time = 0.03's, 88.002'MB/sec, value only(98.1% of KV) wait indexing time = 0.00's, remap KeyValue time = 0.00's, 296.113'MB/sec (all stages of remap) Get OrderMap time = 0.00's, 36.480'MB/sec (index lex order gen) rebuild zvType time = 0.00's, 0.303'MB/sec write SST data time = 0.00's, 1032.763'MB/sec dict compress time = 4356233619.02's, 0.000'MB/sec z-dict build time = 0.00's, sample length = 0.033'MB, throughput = 1099.882'MB/sec zip my value time = 0.03's, unzip length = 0.003'GB zip my value throughput = 88.002'MB/sec zip pipeline throughput = 88.002'MB/sec entries = 2629 avg-key = 20.00 avg-zkey = 0.32 avg-val = 1016.87 avg-zval = 329.31 usrkeys = 2629 avg-key = 12.00 avg-zkey = 0.32 avg-val = 1023.87 avg-zval = 329.31 TagRS{ kinds = RS_Key0_Tag1 bytes = 0 } TagArray{ size = 2629 bytes = 8224 } seq expand size = 18403 multi value expand size = 0 cnt WriteAppend = 11 UnZipSize{ index = 0.0000 GB value = 0.0027 GB dict = 0.03 MB all = 0.0027 GB } __ZipSize{ index = 0.0000 GB value = 0.0009 GB dict = 0.03 MB all = 0.0009 GB } UnZip/Zip{ index = 37.9183 value = 3.0879 dict = 1.00 all = 3.0006 } Zip/UnZip{ index = 0.0264 value = 0.3238 dict = 1.00 all = 0.3333 } ---------------------------- total value len = 11.763792 GB avg = 0.213 KB (by entry num) total key len = 1.497823 GB avg = 0.027 KB total ukey len = 1.032079 GB avg = 0.015 KB total ukey num = 0.068400520 Billion total entry num = 0.055208249 Billion write speed all = 0.325273722 MB/sec (with seq num) write speed all = 0.314440777 MB/sec (without seq num) 2024/11/23-20:14:42.854148 139957634799168 [compaction_job.cc:1948] [default] [JOB 34954] Generated table #278867: 2629 keys, 908476 bytes, temperature: kWarm 2024/11/23-20:14:42.854195 139957634799168 EVENT_LOG_v1 {"time_micros": 1732364082854169, "cf_name": "default", "job": 34954, "event": "table_file_creation", "file_number": 278867, "file_size": 908476, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 4205714669, "largest_seqno": 4236508549, "table_properties": {"data_size": 865744, "index_size": 832, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 0, "raw_key_size": 52580, "raw_average_key_size": 20, "raw_value_size": 2673355, "raw_average_value_size": 1016, "num_data_blocks": 1, "num_entries": 2629, "num_filter_entries": 0, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 20, "fixed_value_len": 18446744073709551615, "filter_policy": "", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "nullptr", "prefix_extractor_name": "nullptr", "property_collectors": "[Rdb_tbl_prop_coll_factory]", "compression": "FlatZip", "compression_options": "RS_Key0_Tag1-UintIndex_IL_256_32-DictZipBlobStore", "creation_time": 1732364082, "oldest_key_time": 0, "file_creation_time": 1732364082, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d89b4006-2803-486e-b7a2-759bcf348921", "db_session_id": "U1ECJGOXSO4Y1P7ZZXV1", "orig_file_number": 278867, "seqno_to_time_mapping": "N/A", "__indexstats__": "[...1 records...]"}} 2024/11/23-20:14:42.862466 139957634799168 [compaction_job.cc:703] job-34954: subcompact[0], size: 0.935571 G, files: /storage/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/mytopling/.rocksdb/job-34954/att-00 [278845,278846,278847,278848,278849,278850,278851,278852,278853,278854,278855,278856,278857,278858,278859,278860,278861,278862,278863,278864,278865,278866,278867] 2024/11/23-20:14:43.002699 139957634799168 (Original Log Time 2024/11/23-20:14:00.434892) EVENT_LOG_v1 {"time_micros": 1732364040434860, "job": 34954, "event": "compaction_started", "compaction_reason": "FilesMarkedForCompaction", "files_L2": [278840], "files_L3": [278760, 278761, 278762, 278763, 278764, 278765, 278766, 278767, 278768, 278769, 278770, 278771, 278772, 278773, 278774, 278775, 278776, 278777, 278778, 278779, 278780, 278781], "score": 0.989399, "input_data_size": 967382433, "oldest_snapshot_seqno": 4238025409} 2024/11/23-20:14:43.002703 139957634799168 (Original Log Time 2024/11/23-20:14:43.001069) [compaction_job.cc:2022] [default] [JOB 34954] Compacted 1@2 + 22@3 files to L3 => 935571484 bytes 2024/11/23-20:14:43.002704 139957634799168 (Original Log Time 2024/11/23-20:14:43.002629) [compaction_job.cc:1224] [default] compacted to: files[0 0 0 23 0 0 0] max score 0.03, MB/sec: 22.8 rd, 22.1 wr, level 3, files in(1, 22) out(23 +0 blob) MB in(31.2, 891.4 +0.0 blob) out(892.2 +0.0 blob), read-write-amplify(58.2) write-amplify(28.6) OK, records in: 2879371, records dropped: 29286 output_compression: Snappy 2024/11/23-20:14:43.002708 139957634799168 (Original Log Time 2024/11/23-20:14:43.002673) EVENT_LOG_v1 {"time_micros": 1732364083002654, "cf": "default", "job": 34954, "event": "compaction_finished", "compaction_time_micros": 42427578, "compaction_time_cpu_micros": 8460683, "output_level": 3, "num_output_files": 23, "total_output_size": 935571484, "num_input_records": 2879371, "num_output_records": 2850085, "num_subcompactions": 1, "output_compression": "Snappy", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "file_write_nanos": 691966453, "file_range_sync_nanos": 0, "file_fsync_nanos": 15950, "file_prepare_write_nanos": 50351, "lsm_state": [0, 0, 0, 23, 0, 0, 0]} 2024/11/23-20:14:43.141124 139957634799168 DEBG dcompact_worker.cpp:240: HttpPost: 200 OK: url = http://nlb-7zny4kaa1poal01o64.cn-shenzhen.nlb.aliyuncs.com:8443, body = {"instanceId":"nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137","labourId":"","dbId":"d89b4006-2803-486e-b7a2-759bcf348921","dbStarts":1732256054,"starts":1732364040,"executesMs":42718,"compactionJobId":34954,"attempt":0,"compactionInputRawBytes":2985895665,"compactionInputZipBytes":967382433,"compactionOutputRawBytes":0,"compactionOutputZipBytes":935571484,"headers":{"Accept":"*/*","Connection":"close","Host":"aliyun-topling-saas.topling.cn:18443","RemoteIp":"120.24.192.169","X-Forwarded-For":"120.24.192.169","region":"cn-shenzhen","token":"e6474c481bd4415abc537135cc03626b","instance-token":"e6474c481bd4415abc537135cc03626b","provider":"aliyun","CloudMarketInstanceId":"si-59c0ae5c955845588903","MeteringAssit":"","Transfer-Encoding":"chunked","traceparent":"00-38be5c37949000cacee51c1b2edca14c-b40a16a78aa3610a-00"}}, response = 2024/11/23-20:14:43.141220 139957634799168 INFO dcompact_worker.cpp:1459: finish /worker/dcompact-log/nfs/i-wz9deglxdazgiryp6ltb_10.57.5.137/2024-11-22T14.14.14/.rocksdb/job-34954/att-00: olev 3, work 42.719 s, result 6.028 ms, install 1.656 ms, input{raw 2.781 GiB zip 922.568 MiB}